ShipTalk Season 4 Finale: Engineering Excellence at AWS re:Invent

1 - >

Speaker 4: Good morning, good afternoon, good evening, time 2 - > appropriate greetings. 3 - > My name is Dewan Ahmed, host of Ship Talk Podcast. 4 - > And this is not just any episode, this is the finale of 5 - > season four. 6 - > And with me, I have Thomas Dockstader, the man in the 7 - > middle of all the madness of reInvent. 8 - > And we can't wait to hear from Thomas what was the discussion 9 - > around uh AI meeting software delivery, which is the theme of 10 - > season four. 11 - > But this time, this time we have a spin with engineering 12 - > excellence. 13 - > Welcome, Thomas. 14 - > Thanks for having me. 15 - > Thank you. 16 - > Tell me what was what was the vibe like? 17 - > Like I heard people were saying like they had like 20,000, 18 - > 30,000 steps uh during reInvent, uh, with with all the the 19 - > energy or the innovations. 20 - > Uh what was it like around the booth? 21 - >

Speaker 7: Yeah, I mean, I think reInvent is, I've been going to 22 - > um trade shows for years and years, and I think reInvent has 23 - > just been one of the ones that has astonished me the last 24 - > couple of years. 25 - > The number of people that are there, the um the lengths at 26 - > which AWS goes to in order to put the show on is incredible. 27 - > Uh it spans across many hotels and just the uh the the trade 28 - > show floor in and of itself. 29 - > I've just never been at a show where there's so many people. 30 - > And when you throw on the um the buzz of AI right now in the 31 - > tech industry uh is just uh a formula for uh a lot of 32 - > excitement and confusion and uh questions. 33 - >

Speaker 4: So yeah, and one of the key questions, I guess, is 34 - > engineering excellence. 35 - > So is this a goal or a struggle? 36 - > How do you see customers uh checking engineering excellence 37 - > as their as their goal or struggle? 38 - >

Speaker 7: Yeah, I would say um, you know, I think really it 39 - > comes up in in two different areas, and and I feel like it's 40 - > becoming somewhat of a pressure point. 41 - > Um, you know, the pulse at reInvent was not so much in the 42 - > in the realm of is AI coming or when is it going to come? 43 - > It's it's here. 44 - > And um I think one of the things that's really being 45 - > magnified is when we talk about code assists and code assist 46 - > being really, in my opinion, the first real big insertion of AI 47 - > into engineering. 48 - > Um, the amount of work that's coming out of code assist, the 49 - > amount of code that's being pushed is accelerated by a 50 - > massive amount. 51 - > And what we're seeing really is that um engineering excellence, 52 - > you know, uh what I heard was, you know, people weren't 53 - > describing it as this nice to have aspiration. 54 - > It's really kind of exposing um some of the areas in the SDLC 55 - > that are not prepared for the kind of influx in code that's 56 - > happening. 57 - > And so because of AI, you know, you can absolutely uh it can 58 - > help teams move faster, by no doubt. 59 - > However, um, if the system underneath is weak and if the 60 - > governance is weak and if the standards are inconsistent, um, 61 - > you know, AI doesn't solve for that. 62 - > It's it's really not that simple. 63 - > And we we we don't have AI inserted into all of the SDLC. 64 - > And so I would say engineering excellence was showing up really 65 - > both as a goal and a struggle. 66 - > Um uh it's really becoming the difference between teams that 67 - > actually can turn AI AI on and into a reliable outcome versus 68 - > the ones that are creating um more more risk and uh you know 69 - > uh more challenges. 70 - >

Speaker 4: Couldn't agree more. 71 - > And you interviewed five uh partners, customers, engineering 72 - > leaders in the space. 73 - > Was there a common thread, Thomas, around how teams are 74 - > using AI to transform their software delivery? 75 - >

Speaker 7: Yeah, I would say that um there, you know, the 76 - > common thread was honestly a little bit more grounded than um 77 - > a lot of the market hype. 78 - > Uh I think that really the shared theme was AI is not 79 - > really um, at least at this point, replacing engineers, and 80 - > more as it's kind of removing friction around engineering. 81 - > And uh it is helping teams move faster, especially with 82 - > repetitive work, test creations and pipeline configurations um 83 - > that can drag. 84 - > And really, I mean, when you talk about coil in engineering, 85 - > um, AI is really great at reducing some of those um uh 86 - > monotonous tasks that we have in engineering. 87 - > However, um, you know, I think one one of the more important 88 - > things was the common thread was that um the the the output is 89 - > increasing and it is creating a tremendous pressure downstream 90 - > uh in the SDLC. 91 - > And so um I think the real transformation is not just you 92 - > know AI making teams faster, uh, it is also forcing 93 - > organizations to mature and create an operating model around 94 - > delivery. 95 - > Um that's that was that was abundantly clear um across all 96 - > of my conversations. 97 - > And that was probably the biggest thing that came out of 98 - > all those conversations was AI is useful, AI is real, and uh, 99 - > but it's already creating that um visibility into areas where 100 - > our uh SDLC might need additional work uh to really 101 - > leverage it. 102 - > And I I could see some organizations um dealing with a 103 - > lot of um tech debt in the near future. 104 - >

Speaker 4: AI is real. 105 - > AI is helping us create test cases for our code, yeah, he's 106 - > deleting our entire mailbox. 107 - > AI is very real. 108 - > Um now, was there a difference in the discussion on how vendors 109 - > are providing AI technology versus customers actually using 110 - > it to ship real products? 111 - >

Speaker 7: Yeah, I would say um, you know, um I think partners 112 - > and customers, you know, they're looking at the same shift 113 - > really, uh, real and and from but really from different 114 - > altitudes. 115 - > Um, you know, partners were generally very focused on what 116 - > AI can unlock. 117 - > Um, you know, and they talked a lot about acceleration and 118 - > automation and uh reducing cognitive load and creating more 119 - > streamlined developer experience. 120 - > I think we see a lot of um a lot of vendors, you know, 121 - > everybody has AI in their name. 122 - > All the tools have AI. 123 - > Um, but uh, you know, where is it actually making marked 124 - > movement in in solving real problems? 125 - > Um, you know, I think uh for the customers, you know, um it 126 - > was much more grounded in operational reality in just kind 127 - > of saying, like, hey, you know, that's great on these features 128 - > that you're talking about and and and everybody's talking 129 - > about them, but how do we validate this? 130 - > Like, where does the value come from? 131 - > How do we validate the value? 132 - > And I think that's a really, really important piece is, and 133 - > we're seeing it in SaaS in general. 134 - > It's like, you know, the idea of moving from a seat-based cost 135 - > model to a um outcome-based um model. 136 - > And I think customers are going to be more and more um 137 - > demanding when it comes to, hey, you've got all these features, 138 - > you've got this amazing AI tool. 139 - > Show me the outcome. 140 - > And that seemed to be pretty consistent in from the customer 141 - > side. 142 - > And so, really, the strongest signal for me uh was that the 143 - > gap between experimentation and production uh is still very 144 - > real, right? 145 - > We we I don't think we've come crossed that chasm. 146 - > Um, a lot of organizations know they need to move to AI, but uh 147 - > I think they're still figuring out how to do it in a way that 148 - > is governed, repeatable, and sustainable. 149 - >

Speaker 4: Yeah, and totally. 150 - > And like these organizations, they're not in uh in a same 151 - > maturity level in terms of their delivery as well, right? 152 - > It's like some are still in like legacy technology, uh, some 153 - > are still like uh uh trying to figure out like how to 154 - > modernize, and while they are trying to do that, now there's 155 - > this AI transformation they have to do. 156 - > So, uh did did you talk to some customers uh uh trying to now 157 - > battle like two grounds. 158 - > One is modernizing their legacy tech, and now they have to do 159 - > AI modernization as well. 160 - >

Speaker 7: Yeah, it's interesting. 161 - > Um, you know, I do uh in addition to you know reInvent, I 162 - > do SDLC assessments for organizations, and it's funny 163 - > because as fast as AI is moving, um I don't know that there's a 164 - > solve for how we we move a legacy system. 165 - > You know, it there's just so much data, and there's so many 166 - > um uh learned and baked-in processes within an organization 167 - > that um and there's a lot of um individual knowledge that's 168 - > held by the individuals who who do those jobs, not documented 169 - > necessarily in a system that AI could go in and read. 170 - > And so I think it's gonna be a combination of um someone's 171 - > gonna have to come up with a pretty smart idea on how to do 172 - > this, but it's gonna be challenging for those legacy 173 - > systems to move. 174 - > I think some of them will get um could get disrupted, quite 175 - > honestly. 176 - > Um that's a that's a big possibility in uh um being on 177 - > some of those older systems and just not having the ability to 178 - > move to a velocity space. 179 - >

Speaker 4: Yeah, and I'm pretty sure like we'll have some more 180 - > insights from these five interviews. 181 - > So without further ado, let's listen to the interviews. 182 - >

Speaker 7: Thank you for coming out. 183 - > Thank you for happy to have you in in the booth to record that 184 - > to record uh an episode with you. 185 - > Um this show's crazy. 186 - > It is, and um, there's a lot of I see one word or sorry, two 187 - > letters, AI is everywhere, right? 188 - > Sure is. 189 - > It's uh it's a pretty popular uh subject. 190 - > It's what I write about all the time. 191 - > And so I'm I'm so interested to hear your uh opinion on things. 192 - > I really want to understand like what are you hearing um 193 - > that that that AI is actually helping versus kind of hype not 194 - > actually doing much? 195 - > Right. 196 - > Well, it's not an easy question to answer, right? 197 - > Let's go simple, yeah, yeah. 198 - >

Speaker 6: Some simple ideas. 199 - > I think that um, you know, everybody I talk to, and I talk 200 - > to a lot of CIOs, CTOs, sense of AI, obviously, people know that 201 - > they kind of have to be getting the companies moving in this 202 - > direction. 203 - > Um you can't go from like zero to a hundred. 204 - > You know, you have to kind of there's things you have to do 205 - > first. 206 - > So one of the big things is data, right? 207 - > You have to get your data in order, and then you can begin to 208 - > start to take advantage of this stuff as you tune the models to 209 - > your data. 210 - > But people know that they have to go there, not everybody is 211 - > there yet, and there's a lot of experimentation that's going on. 212 - > Um, what I'm hearing is that there's less like in production, 213 - > but more um, you know, we're still doing proofs of concept, 214 - > we're still doing experimentation. 215 - > We're trying to figure out like how do we use AI to really like 216 - > improve efficiency to get that return on investment. 217 - > But, you know, being a um a company that deals with 218 - > developers, one of the areas that seems to be the most um 219 - > mature is the code area. 220 - > So that's an area where you see a lot of companies kind of 221 - > diving in a little deeper than maybe some of the other areas. 222 - >

Speaker 7: Yeah, I think it's funny you mention that because I 223 - > feel like that portion of the SDLC is accelerating quite a 224 - > bit. 225 - > Yeah. 226 - > My assumption is well, it's actually happening because we're 227 - > seeing that deploy deploy times are higher because there's so 228 - > much code coming in, the PRs are accelerated, everything's going 229 - > faster. 230 - > I really wonder what's going to happen, call it a year from 231 - > now, if we're gonna have this tidal wave of bugs, right? 232 - > And like the the ability to fix them, I think because maybe the 233 - > developers aren't so familiar with the code like they were, 234 - > you know, 20 years ago when John, who has been with the 235 - > company for 20 years and he knows the code backwards and 236 - > forwards, if we're throwing a bunch of code in there, you 237 - > know, it could potentially cause additional problems. 238 - > I'm curious, the leaders that you're talking to, are they are 239 - > they thinking about this? 240 - >

Speaker 6: Or I mean, I think people have to be thinking about 241 - > that, right? 242 - > Because one thing that AI allows you to do, as you said, 243 - > is to go faster. 244 - > Yes. 245 - > Like going faster has its own danger with it. 246 - > So there are two things here, I think. 247 - > One is your experience development team was going to be 248 - > using these tools to kind of do things that were kind of 249 - > routine and not very fun to do, right? 250 - > To get those things done faster without kind of manually going 251 - > in and doing them. 252 - > But you also have this idea that maybe the citizen developer 253 - > can start to develop. 254 - > And I I think that's where a lot of problems could start to 255 - > prop up because people who don't know the kinds of things that 256 - > an experienced engineer knows can get into trouble very 257 - > quickly, and they don't know security, they don't know 258 - > governance, and they're just like, oh, look, I can create 259 - > this program. 260 - > Um, and I think there's part of the problem is that the 261 - > engineering team is gonna have to reel that in. 262 - > That's that's that's one side of it. 263 - > But the other side of it is like, how good is this code? 264 - > And you're putting this code into your pipelines, right? 265 - > And like you said, you don't necessarily know it. 266 - > I'll tell you a quick story. 267 - > I was in Miami a few weeks ago, and I was in a coffee shop 268 - > getting coffee before my meeting in the morning, and I see these 269 - > two guys, I overheard these two guys talking. 270 - > One guy's like sitting there, he's bleary-eyed, and his 271 - > buddy's like, Wow, you look tired. 272 - > What's going on? 273 - > And he's like, Well, I had to read 10,000 lines of code last 274 - > night. 275 - > And the guy's like, Can't you get AI to do that? 276 - > And he's like, No, I have to know my code. 277 - > So, what you said, like like there's still, even though 278 - > you're creating it faster, maybe that even makes it harder. 279 - > Because if you're somebody who wants to still have that sense 280 - > of, I need to understand what's going in my pipeline, then 281 - > that's not and sure, you're efficient on one hand, but on 282 - > the other hand, you're like, now I got all this code, I have to 283 - > be uh supervising. 284 - >

Speaker 7: How do you think um companies are redefining 285 - > engineering excellence because of AI? 286 - > Like, what's like what how what are you what are you seeing 287 - > there from trends? 288 - > Because I know a couple of my guests have talked about center 289 - > of excellence uh groups and how they're so needed in the or to 290 - > your point, creating compliance and um making sure we have 291 - > checks and balances and all those things. 292 - > Are the are the higher ups thinking about this from a 293 - > perspective of I I think they have to be, right? 294 - >

Speaker 6: You know, um and like I I spoke to um the the guy who 295 - > runs the uh the center of of innovation here at AWS 296 - > yesterday. 297 - > Okay, so they are working you know directly with customers to 298 - > you know kind of help them understand everybody's kind of 299 - > learning together, right? 300 - > I think the vendors are learning, the developers are 301 - > learning, the companies are learning, and it's it's this 302 - > kind of chaotic mix in a way, because everything's happening 303 - > so quickly. 304 - > And I think back like a year ago, like there was no there was 305 - > no um you know management layer, there was right, there 306 - > was very little security. 307 - > Yeah, um, you're starting to see things like that be 308 - > announced um you know by AWS and others. 309 - > Um and like when you think about something like an agent, 310 - > it needs basic stuff, you know, it it needs all the stuff that 311 - > you've always needed, but as you say, you're making it 312 - > everything faster and faster. 313 - > So you have to have those checks and balances in the 314 - > pipeline, or things are gonna blow up when you're pretty fast, 315 - > I think. 316 - >

Speaker 7: Yeah, that that actually leads me to my next 317 - > question is like, what do you think the skills are that that 318 - > that leaders are looking at to really develop? 319 - > And and and I with my last guest, we were talking about 320 - > agents. 321 - > And I want I I asked the question are we are we now 322 - > looking for talent that is learning specifically how to 323 - > manage agents alone? 324 - > Like, what is what is the trend from a perspective of the type 325 - > now? 326 - > You also mentioned experienced developers. 327 - > I've heard that trend where uh some organizations are leaning 328 - > more towards the heavily experienced developers versus 329 - > hiring new out-of-college developers. 330 - >

Speaker 6: Where so I think there's two schools of thought 331 - > on that. 332 - > Some people are saying we're hiring a lot of young people 333 - > because they get this stuff apparently, right? 334 - > But you know, I think ultimately you need both, right? 335 - > Yeah, you still need mentors, you still need like even if 336 - > you're an old school developer and you're working with a young 337 - > kid out of college, you still need to um you know have that 338 - > mix of skills and understanding how things get built, right? 339 - > Um in terms of skills, I think I I wrote I wrote a commentary 340 - > um earlier this year that was it was kind of interesting to me 341 - > because when you think back, engineers become engineers 342 - > because they're good at math, they're good at logic, they're 343 - > good at understanding kind of like how code fits together. 344 - > But when you start to become AI driven, the importance of 345 - > prompting becomes you know, a lot becomes more important. 346 - > And yeah, that's true. 347 - > You know, so then what is what is that skill? 348 - > That's writing, right? 349 - > It's writing and describing what you want. 350 - > And if you're building in an agent, like it's one thing to 351 - > prompt, you know, a chat GPT type model and say, I need this, 352 - > and that's you know, I'm a writer and I I find like you 353 - > have to be kind of like play with them until you get what you 354 - > want out of it, right? 355 - > When you're an engineer and you're creating an agent that 356 - > may be something you use internally or something you're 357 - > selling to customers, that prompt has to be really rich. 358 - > Right. 359 - > It has to really have a you have to have a lot of expertise, 360 - > and you have to be able to articulate that expertise. 361 - > So suddenly I think it's no longer just math and logic, 362 - > although I think that still applies. 363 - > Suddenly you need those English skills. 364 - > Yeah, the skills, the skills that I have suddenly become, you 365 - > know, people say, oh, you know, with with uh these large 366 - > language models, writing becomes less important because models. 367 - > I don't I don't agree with that as a writer, but I also think 368 - > in some ways it becomes more important because you have to be 369 - > able to communicate what you want to these models. 370 - > And if you're building, you know, an automation agent that's 371 - > going to do a lot of stuff, yeah. 372 - > You have to first of all understand that process inside 373 - > and out, and then you have to be able to articulate it in a way 374 - > that you can communicate it so that the models can carry out 375 - > these identic tasks, right? 376 - >

Speaker 7: That is a very important point, and I think 377 - > it's it's kind of scary because my son got in trouble for using 378 - > AI for his paper report, right? 379 - > So maybe the the coming generation isn't developing 380 - > their writing skills because of AI when in reality we need it, 381 - > right? 382 - >

Speaker 6: We absolutely need it, and and I think that um your 383 - > son and all of all the people in that you know in our younger 384 - > generation have to kind of learn both, right? 385 - > Yes, I mean because like I don't use AI to write, but I I 386 - > use AI to edit, you know. 387 - > So I'm a I'm a I run that newsletter and blog that I have 388 - > by myself. 389 - > So I have a I have a human editor that that checks the 390 - > pieces, but as I'm writing, I'm like I'm checking it against 391 - > like, you know, did I make any mistakes? 392 - > Did I, you know, did I overly repeat words? 393 - > Did I um that kind of stuff is really nice? 394 - > Does it thematically hold together? 395 - > You know, is it is it logical? 396 - > You know, and you can ask the models these things, and it's 397 - > like having this editor, but even with that, like when I give 398 - > it to a human editor, she always finds stuff and she'll 399 - > say, like, I don't know what you were trying to say here. 400 - > Why does this have m dashes? 401 - > Those cursed m-dashes. 402 - > I mean, I've used m-dashes before, but they those models, 403 - > they just love that. 404 - >

Speaker 8: Oh, I don't know why. 405 - >

Speaker 7: And it's an instant, now it's an instant indicator. 406 - > Oh, it's a test written with an AI. 407 - > So um how do you think, how do you how do you think in 408 - > enterprises are measuring this? 409 - > How are they measuring AI? 410 - > Uh whether it be code or have you heard any trends or anything 411 - > that's coming around from perspective of how they're 412 - > actually going to measure this? 413 - >

Speaker 6: I mean, I I I think even in the past, you know, 414 - > sometimes they would say, like, you're a productive engineer 415 - > because you've produced X lines of code per day, per week, per 416 - > month, right? 417 - > Um I've heard people say that even that is pretty flawed. 418 - > look at engineering excellence so um but if you think about 419 - > that as a metric that becomes less important right because if 420 - > you can tell the model what you want and the model produces a 421 - > bunch of code for you and then you massage that code and then 422 - > find ways to you know mix that with other things that you're 423 - > doing that you've created then where's the where's the 424 - > productivity measure yeah right so um I think companies have to 425 - > start thinking about how they they they measure uh the 426 - > production and efficiency of engineering moving forward 427 - > because you know when you're gonna be working with ai as a as 428 - > kind of a a worker and I kind of I don't really like that that 429 - > metaphor yeah but I mean you are going to be working 430 - > alongside this entity yeah that's very true and um 431 - > companies have to find ways to say well this person you know 432 - > knows how to use ai really well and you know that has made them 433 - > more productive uh you know this person maybe is doing uh more 434 - > manual coding but it's better code you know so so I mean like 435 - > you have to kind of start to balance all that stuff and I 436 - > don't think it's one or the other right I think we're gonna 437 - > whether it's writing or engineering or whatever it is 438 - > it's we're gonna find we're gonna learn how to use this 439 - > stuff uh as a tool and I think it is it's a tool very much so 440 - > in in your quiver that you know um helps you do your job but it 441 - > doesn't do your job right yeah that's a great point yeah as a 442 - > reporter and a writer with all of this you know I mean look 443 - > every company's trying to get attention right because they're 444 - > they're they're implementing AI they're adding something new how 445 - > do you specifically determine what stories you chase after I 446 - > mean you know the fundamentals of what's news don't change you 447 - > know okay there's always a technological trend that comes 448 - > down the bike mobile cloud you know cloud yeah I mean the 449 - > internet going back you know yeah I mean so uh you know I 450 - > mean and I I've been around that long but uh you know what was 451 - > important in the early 2000s when you went to a conference 452 - > like that was this was very different right yeah sure um and 453 - > and I mean over the years everything kind of shifts you 454 - > know if you came here in 2018 probably like a lot of people 455 - > talking Kubernetes and cloud native yep um you know you came 456 - > here in 21 to 22 people talking RPA and that that kind of 457 - > automation and now here we are in 25 and it's just all AI all 458 - > the time. 459 - > And I would be lying if I didn't say that had my 460 - > attention. 461 - > I mean I write about it a lot um but everything that has AI 462 - > attached to it is not interesting just like every 463 - > other wave that came before it um as a journalist I have to use 464 - > my judgment which I think AI can't do and it is my special 465 - > sauce as a human is that I can look at something and based on 466 - > my experience and what I find interesting say I think this is 467 - > newsworthy or this isn't and or I've seen this 10 times and I 468 - > don't want to write about it again. 469 - >

Speaker 7: You know yeah you're actually in a unique position 470 - > because you do get to see kind of like early you know early 471 - > announcements or people kind of reaching out to you hey we're 472 - > doing this we're doing that you probably get to pick out those 473 - > eight well actually 12 people are doing that it's not that 474 - > unique right they they may not know that right that's very 475 - > interesting. 476 - >

Speaker 6: Yeah I mean and from whether it's a startup 477 - > perspective or an established company um you know there are 478 - > there are always I'm always looking for like you know have I 479 - > seen this before if I haven't seen it before and it's solving 480 - > a problem that I hear about you know then that's something 481 - > that's gonna kind of pique my interest and I'm gonna want to 482 - > dig into it and write about it and see if it's actually a trend 483 - > that's developing. 484 - > You know so um what is news is like that's always gonna be like 485 - > okay what do I find interesting is my own unique filter but 486 - > that doesn't change because of AI. 487 - >

Speaker 7: Sure. 488 - > Yeah that makes sense that's a great point. 489 - >

Speaker 6: Yeah you're right the internet was a pretty big thing 490 - > pretty big deal when it came out the web you know I remember 491 - > the web I remember when you know in the late 90s and early 2000s 492 - > when companies were trying to decide whether they should have 493 - > a web presence or not you know and they didn't know what it was 494 - > they didn't know how to deal with it. 495 - > And you know obviously that that changed in heart. 496 - > Yeah now though everything's operated on that now I think you 497 - > know we see all these companies struggling with AI but 498 - > eventually it's gonna be like SaaS right it's like you don't 499 - > look at a software company and say oh they're SaaS now right 500 - > you know like every company is is software. 501 - > It's what it is yeah right so I I think with AI AI is going to 502 - > be baked into everything and it may be not something that we 503 - > talk about it as as much as we do now. 504 - > Right. 505 - >

Speaker 7: So with your with your you know um insights into 506 - > the business all over the place what what is something that is 507 - > generally exciting to you um that's coming that you see well 508 - > I mean I think that as agents develop you know there's there's 509 - > there's this whole infrastructure around it that 510 - > has to develop that we're only beginning to see around security 511 - > around governance around all the fundamentals of IT right 512 - > like whatever it is whatever type of software it is you can 513 - > call it whatever you want today it's agents there are certain i 514 - > and I I wrote a piece um about this at one point where the 515 - > fundamentals still apply right you have to be fundamentally 516 - > sound in a large organization especially or you know it's just 517 - > not gonna work so I think we're starting to see some of that 518 - > infrastructure develop and I mean I was at RSA in April right 519 - > in big security conference in San Francisco and there were 520 - > people talking like you know these things are going to have 521 - > to go out communicate with other agents cross systems maybe I 522 - > mean we don't know how interesting the vision will 523 - > match the reality but as those things happen um you know you're 524 - > gonna need a unique kind of security right because these 525 - > things are gonna go do something they're gonna become something 526 - > else and then maybe their their identity and authentication 527 - > suddenly are different and they're in their neural interest 528 - > and like it used to be that software was pretty linear right 529 - > you know so those ones and zeros yeah so so so so as you 530 - > went through that process it was fairly easy to secure it once 531 - > you secured it right now it becomes a little bit more loosey 532 - > goosey because um things are changing so much and these 533 - > things can be you know they can move and they have what they're 534 - > meant is that they're highly flexible right but their their 535 - > danger is that they're not flexible like how do you secure 536 - > something that can change and so that that I find pretty 537 - > fascinating. 538 - >

Speaker 3: It's exciting I'm I'm telling you what um we're gonna 539 - > do a little segment ship it or skip it okay um office pets um 540 - > ship it ship it all right I love that uh four day work weeks um 541 - > ship it yes thank you once you once we get to four then I'll 542 - > push for three yes with AI right with agents we can we could get 543 - > there yeah I mean I mean if you're gonna have AI and it 544 - > truly makes us more efficient why not give it it's supposed to 545 - > give you time yes and you hear some people measure it that way 546 - > you know like it can't it it saved eight hours of development 547 - > it saved eight hours of research whatever it is like why 548 - > not give that to us right yeah absolutely I couldn't agree with 549 - > you more yeah well hey I really appreciate you coming on thank 550 - > you for having thanks for uh love it love your opinions and 551 - > uh enjoy the show thank you thanks hey thanks for coming out 552 - > to reInvent my gosh this place is crazy my pleasure it's I mean 553 - > my feet are killing me I mean I've been yeah no kidding right 554 - > yeah I'm doing more than like three 30 000 steps so yeah 555 - > that's what I love about Vegas it's like oh it's just next door 556 - > that's like five miles away yeah it's incredible these these 557 - > these convention centers are absolutely massive they just 558 - > keep going forever and everyone um I'm I'm happy to have you on 559 - > today I would love to get your perspective on a couple of 560 - > questions um the first one is I'm just interested to 561 - > understand from your perspective what do you think right now is 562 - > hype and real with AI in technology like where do you see 563 - > some good and some maybe so uh I think AI existing AI can be 564 - > applied to technology in so many ways to uh reduce soil and 565 - > accelerate certain things uh where it's uh high high phase uh 566 - > really anything around uh um uh what's it um uh AI replacing 567 - > humans completely and uh kind of uh AGI okay that's that's high 568 - > because uh you know fundamentally these systems are 569 - > still uh you know next token prediction that type of thing 570 - > yeah very infant level yeah yeah but but what they are really 571 - > good at is uh with the data it has uh it's able to process a 572 - > lot of things very quickly uh and and appear intelligent uh 573 - > and and deliver outcomes that way and there's a lot of 574 - > transformation you can do i mean even even with the existing 575 - > technology if he can productize things I think we we easily have 576 - > more than 15 years of uh transformation that can happen 577 - > and and deliver a significant value for the society oh 578 - > absolutely that's great so when we talk about platform speed and 579 - > security where do you um especially with AI yeah where do 580 - > you draw the line because I feel like code assist is really 581 - > speeding up velocity but maybe causing security issues. 582 - > Yeah I I think the important thing is uh I think doing 583 - > platform and coding right is important to get speed uh long 584 - > term and that's where I think uh you have uh uh security angle 585 - > also fashion sometimes. 586 - > Uh I mean I I use this uh term uh frictionless security and uh 587 - > it often boils down to make sure that you make it easy to do the 588 - > right thing and that way you go a lot faster versus uh blanket 589 - > security policies that slows things down and uh frustrates 590 - > everybody uh and and people often looking for a workaround 591 - > because uh security is in the way and that in fact uh lead to 592 - > poor security and slowing things down versus uh good security at 593 - > speed. 594 - >

Speaker 7: Are you seeing AI improve that process of getting 595 - > security approvals and like moving quicker? 596 - >

Speaker 3: Definitely AI can uh help in so many angles so things 597 - > like uh you know compliance checks uh you know security has 598 - > a ton of uh oil work uh and continuously testing things and 599 - > uh validating certain things I think uh those type of things AI 600 - > could really help I feel like I feel like security always takes 601 - > it just takes so long especially like if we're trying 602 - > to sell some software yeah right it takes so long what what 603 - > where do you see that where do you see the future of it I mean 604 - > in the next let's say 18 18 months to two years the because 605 - > uh it takes so long because of uh certain regulations and uh 606 - > things that you need to comply and then more often uh it's 607 - > because of the approach that people have taken why it takes 608 - > so long uh you know you usually put a human in the mix to the 609 - > process uh and then most of the time uh you do it because uh you 610 - > know these approval chains uh make you feel good about it but 611 - > not necessarily sometimes uh actually doing the right uh type 612 - > of security uh things you know even in uh compliance you know 613 - > you have different personas that you bring in but the exercise 614 - > itself doesn't uh I mean it's better than nothing but that's 615 - > also fundamentally broken in certain ways to uh have like 616 - > real security versus uh a tick box exercise and uh kind of like 617 - > the perception of security right what do you that that's a 618 - > great that leads me into this next question what do you think 619 - > is the most overrated security control that's a human and 620 - > approvals because because what often happens is uh human and 621 - > okay no no no because I'm not saying it's a bad thing sure we 622 - > need it uh but let me give you two angles right uh one is uh 623 - > you put a human in the loop uh where you think human is gonna 624 - > make uh better judgment calls and uh do the right thing but 625 - > then you overwhelm that um uh situation and then it becomes uh 626 - > you know a lot of things escape and without uh understanding uh 627 - > that becomes like a uh you know clicking exercise where the 628 - > human doesn't quite understand the implications of everything 629 - > that's going on right and then uh and then that's a fundamental 630 - > failure uh in the process itself uh and then the other 631 - > cases uh when you have uh many stakeholders and approvals you 632 - > you often uh delay things significantly uh without having 633 - > uh kind of exact clarity on uh what is being applied well and 634 - > what are the real implications and how how to kind of like 635 - > proceed with it. 636 - > So yeah where do you think security tools should live in 637 - > the platform team product team in the I would say in the 638 - > platform team primarily uh so I have an interesting role uh I 639 - > have uh I'm platform engineering uh director as well as CSO for 640 - > my group now uh in in one way you can look at empowerment and 641 - > the stake uh of the two roles uh and the nice thing is uh you 642 - > accomplish frictionless security by making it easy with platform 643 - > engineering to do the right thing so more you do that you 644 - > get better security and a better developer experience compared 645 - > to uh just uh you know uh security for the sake of it or 646 - > security theater right do we feel like okay so do we feel 647 - > like the platform team could potentially not understand 648 - > context of the product teams and there's a little bit of 649 - > friction there like so so there is friction because uh if you 650 - > look at uh like whether it's a you know a security organization 651 - > or a platform organization these things are fundamentally 652 - > bottlenecks that are put into any organization to uh they're 653 - > good bottlenecks I'm not saying bad bottlenecks uh they're put 654 - > in to uh streamline the process and make sure that you do things 655 - > sensibly uh so any bottleneck you'll always have frictions if 656 - > things are not running smoothly and competitively so this is 657 - > where I think you need to uh you know it's more people 658 - > relationships understanding empathy and what people like 659 - > really want along with ensuring the process and making sure what 660 - > the the bottleneck works efficiently and offer a 661 - > competitive offering for all the stakeholders I think that's the 662 - > that's the key about an effective uh bottleneck more 663 - > often what happens is in organizations uh uh security 664 - > function uh perform engineering functions they are they're 665 - > fundamentally broken because this uh that the the bottleneck 666 - > is not working efficiently and it's it it over time uh it's not 667 - > competitive enough isn't it true that some long-term 668 - > security employees just like to say no so like I mean uh I mean 669 - > so I'm I'm a CISO and uh you know we have groups right like 670 - > uh there's a funny saying like you know it's the chief uh 671 - > escaped right yeah the so so I think you know that's a very 672 - > interesting uh uh angle sometimes because uh your job is 673 - > on the line yeah exactly when you say when you say yes to 674 - > things right uh and then if you know something really bad 675 - > happens you're you're putting your job in in the line uh so so 676 - > there is an inclination to say no or start with no yeah uh 677 - > without necessarily appreciating uh what the business is trying 678 - > to accomplish and uh looking at more pragmatic parts. 679 - > But yeah I mean I think you know the thing is you know I 680 - > don't know like personally if you ask me uh uh that's uh not 681 - > exactly a career aspiration for me on uh on one side like I'd 682 - > rather think more on hey uh yes but and then try to like that 683 - > get them on uh hey let's do it in a frictionless way do it 684 - > better these are the implications uh yeah I mean it's 685 - > uh I like that yes and actually not even yes but I would say 686 - > yes and you need to do XYZ right. 687 - >

Speaker 7: What is one AI security risk that is real right 688 - > now versus one that's just not really that so much of a worry. 689 - >

Speaker 3: I think uh one that is not real is the hype uh which 690 - > is uh I mean it's almost like uh it's not a problem with uh AI 691 - > uh it's just people misusing AI and just uh creating chaos and 692 - > trouble. 693 - > So I you know that there's a lot of uh drama going on yeah 694 - > around that but uh uh but what's some examples of real things 695 - > are I think uh uh data security is very real so uh whenever we 696 - > look at an uh AI uh tool or anything I think where your data 697 - > flows are and how that gets uh exposed is uh very important to 698 - > think about because uh uh you know with modern systems uh you 699 - > you know you you don't always have control of your data you 700 - > have to sometimes you know trust third parties uh they leave 701 - > certain boundaries so it's really important to understand 702 - > those flows and how you secure it because uh that's an that's 703 - > an area you can get into trouble. 704 - > Another one is uh privilege escalation. 705 - > So we've been doing a lot of uh agent it uh and uh one pattern 706 - > that's been happening is uh when it comes to agent tools you 707 - > give uh service account access to uh sub agents and agents and 708 - > whereas when it comes to the humans you have particular RBAC 709 - > scopes and and uh things around that so mapping those uh roles 710 - > to what it can do I think that's a big big big challenge because 711 - > uh not do getting it right would lead to uh uh you know 712 - > accidental or intentional privilege escalation so that's a 713 - > uh valid challenge that uh we're seeing right now if you 714 - > had to improve developer experience and raise your 715 - > security bar in the same quarter what single move gets whole uh 716 - > it's doing platform engineering right with the security uh 717 - > almost partnering with security teams so wherever so I I mean 718 - > it's nice that I own the both roles right now but uh in the 719 - > cases where I haven't I would uh seek out to the security 720 - > organization partner with them understand their requirements 721 - > and give them also good angles around how a fundamental 722 - > platform engineering move will improve security as well as 723 - > accelerate uh developers making it easy to do the right thing 724 - > versus uh the wrong thing so uh that's that's what I call 725 - > frictionless security and uh you know yeah that's kind of your 726 - > thing yeah yeah is uh is there anything you've seen at the show 727 - > today that is kind of like wow yeah oh that I mean there's 728 - > there's been a lot lots lots of stuff I mean uh I think uh you 729 - > know uh certainly agent AI it's interesting to see how how how 730 - > much is possible and also in some way uh some ways how much 731 - > is not realized right so it's like uh uh that's kind of like 732 - > what's uh uh most interesting so if I'm if I'm like uh when I'm 733 - > thinking about next year uh I think hopefully all of these 734 - > things being possible and actually going into production 735 - > use cases and being realized I think uh that's the that's the 736 - > exciting so do you think agentic is like the next big leap where 737 - > we make a change to your point like where it becomes more 738 - > trustable more you know usable yeah I mean uh so there's a 739 - > couple of things around agent it one is to get around uh LLM 740 - > sort of limitations and uh uh kind of you know say even if the 741 - > foundational models don't uh advance agent can be leveraged 742 - > to solve a lot more complex problems. 743 - > The other one is the data problems we've had for decades, 744 - > right? 745 - > I mean if you look at it uh people still don't have uh their 746 - > data platform problems and other things sorted out and more 747 - > often than not the biggest concerns are around data so 748 - > agent can really help where you can have uh separation of 749 - > concerns with agents and specific data and them 750 - > collaborating with each other so that that's not something that 751 - > we are seeing much right now but I think uh next year perhaps we 752 - > will see a lot of that where you can say you know I'll I'll 753 - > give an example uh you know if I have a personal uh health agent 754 - > I I I will trust my uh personal health data with it but then if 755 - > if that agent is uh interacting with uh different other agentic 756 - > system that agent could be empowered to share what's the 757 - > program than what's not, right? 758 - > So you can you can kind of like uh safeguard certain 759 - > information in that way. 760 - > And then if you look at like the enterprise world and uh data 761 - > silos and all the concerns around data, I think that's a 762 - > really significant piece that would work in favor of agent. 763 - >

Speaker 7: So do you so do you think do you think we need to 764 - > move to a place of a talent base that understands how to run 765 - > agents? 766 - > Like less of an like it's not I guess it is engineering, but 767 - > like it's almost like their job is to run agents. 768 - >

Speaker 3: And also it's uh it's a transformation right so in in 769 - > some ways uh when I when I think about AI agent all of this 770 - > uh uh technical capabilities there they are I think yes there 771 - > has to be a transformation but uh that's ones and zeros and 772 - > binaries in some ways easier right the social and the human 773 - > uh people and the process aspect that's gonna be the more more 774 - > significant thing that needs to be uh transformed so you know 775 - > processes need to evolve be challenged in order to improve 776 - > things and then people they really need to have a uh growth 777 - > mindset around how uh AI can really help because if you start 778 - > with uh AI is gonna take my job that's a non-starter you you 779 - > are uh I don't like you know exactly I mean uh so it's like I 780 - > I can't see AI taking jobs uh like you know in the story in my 781 - > lifetime I like I don't see that right the but it's really 782 - > important leveraging AI in order to do things better I think we 783 - > all have a lot that we can uh kind of like really gain 784 - > absolutely okay we're gonna do a segment ship it or skip it okay 785 - > yeah sure you're right AI code reviews AI code reviews I think 786 - > uh skip it skip it for now for now yeah as again you said we 787 - > just we quite don't quite trust it yeah it just needs to get a 788 - > little bit further sorry sorry AI oh my bad let me go let me go 789 - > AI code reviews AI code review code reviews uh so using AI for 790 - > code reviews yes uh very very worthwhile leveraging because AI 791 - > can uh really summarize very complex uh code reviews it still 792 - > need to be reviewed so like the thing like uh let's take GitOps 793 - > in platform engineering okay GitOps PRs are a joke in most of 794 - > the teams because they are so big and people don't understand 795 - > and like most of the time people just click the button in 796 - > approval right so so it's like a so using AI to help people 797 - > understand it's significantly better. 798 - > And and and the same thing applies to uh code uh I mean in 799 - > some teams you know hey keep it small so people can understand 800 - > like that's been the golden rule if you go a few years back now 801 - > with uh code being generated there's so much that gets 802 - > generated right and and and when you're doing uh reviews yeah if 803 - > you don't use AI it's very difficult to uh understand 804 - > exactly what's being changed and what are the implications so 805 - > it's like yeah that's a bottleneck and I I think it's 806 - > important to leverage AI yeah yeah all right ship it or skip 807 - > it yeah four day work weeks four day work weeks careful careful 808 - > I'd say skip it I mean uh it's like a it's an interesting thing 809 - > I think uh I mean you you have like a so I I've I've led like 810 - > uh uh a lot of different cultures a lot of different 811 - > teams right I think uh depending on where you are like uh you 812 - > have a very different uh mindset and an attitude right I think 813 - > uh to be honest like I'd say like the time you work is 814 - > irrelevant as long as the outcomes are easy right so it's 815 - > almost like uh I don't really care whether people work nine to 816 - > five or uh five days an even better answer right it's like 817 - > what I care about is the outcome right so it's like uh I don't 818 - > want to be a bean counter hey did you clock in clock out or 819 - > are you are you there are you in the office for production yeah 820 - > it's uh it's a what have you accomplished right like uh makes 821 - > sense all right no deploy Fridays keep it that's uh that's 822 - > that's excellent because uh I have done it like I mean it's 823 - > just you know unnecessary stress right I mean uh you're winding 824 - > down you do something bad and then uh you ruin the round 825 - > everybody gets everybody right and the cost of that and then uh 826 - > the toil that creates for the next week it's not worth it so 827 - > it's like better to uh ship uh Monday to Thursday where you can 828 - > actually you know deal with things uh and then uh you know 829 - > relax a little bit more and it it actually helps you go faster 830 - > in the long run all right one more yeah LeBron James I don't 831 - > know thank you sir thank you appreciate it Eric man thank you 832 - > for coming out to uh reInvent it's absolutely wild here 833 - > awesome um the craziest show that I've ever been to right 834 - > yeah this place is just insane bananas i i i this gets bigger 835 - > every year and it's always fun anyway well hey look I wanted to 836 - > have you on because um I'm interested to get your 837 - > perspective on some of the trends that are happening now in 838 - > in the industry yeah and um I think the AI is obviously one of 839 - > the biggest uh hot bucket button topics that we're uh 840 - > hearing and I'm curious to kind of understand from your 841 - > perspective like um basically what are you what are you 842 - > feeling is kind of like hype yeah versus actually going to 843 - > help productivity and and and efficiency. 844 - >

Speaker 1: 100% you know it's interesting I the one of the 845 - > things that in being in this space now I've been in the 846 - > software development space for the last 20 years which is scary 847 - > I got gray in the hair all it's just what it is. 848 - > But the coolest part that kind of happened when a lot of these 849 - > model providers really started to come out was I've got all 850 - > these different line of business use cases that we want to try 851 - > to go build wealth management advisors or trading platforms 852 - > and all that's happening. 853 - > And I think that there's real value that's being derived in 854 - > the creation of those different types of agents and use cases 855 - > with the model. 856 - > But the one that really quickly that I think we've all seen 857 - > truly emerge that went so fast from prompt engineering all the 858 - > way to I'm gonna build a fleet of software engineers as AI is 859 - > in the development space. 860 - > And what has been amazing excuse me what has been amazing 861 - > is there is a whole set of my customer segment I manage the 862 - > global financial services business for our developer 863 - > platform Gen AI segment. 864 - > And seeing how customers are taking advantage of those 865 - > different services to accelerate what the developers are able to 866 - > actually achieve and how fast adoption has happened is unlike 867 - > anything I've ever seen in the 20 years I've been in the 868 - > software development space. 869 - > And I think that where they're seeing the most value is how are 870 - > we able to think about an idea of a feature that I want to go 871 - > build and begin to take you AI to offload all of the remedial 872 - > things that I needed to do, whether it's test creation, 873 - > whether it's pipeline configuration, whether it's 874 - > building the right infrastructure templating 875 - > patterns and the way we're going to deploy it's giving 876 - > developers the chance to try to use it for feature development, 877 - > which is critical, right? 878 - > But the area where we're seeing the most impact is offloading 879 - > all of the things that Plat Engine operations security were 880 - > putting back on development where where you looked at their 881 - > typical day they might have only been writing lines of code on a 882 - > feature for an hour or two and they're able to get 883 - > significantly more freed up in that space. 884 - >

Speaker 7: So it is accelerating significant. 885 - > It's accelerating okay so that's that's a great point. 886 - > So everywhere I look around here everybody's about AI right 887 - > okay so from your perspective what is something that is that 888 - > demos really well or or and actually doesn't really help? 889 - >

Speaker 1: Yeah. 890 - > So I think that if if I've and I've had a good fortune of 891 - > working with lots of different technology providers including 892 - > our own in the space right and that's what something that we at 893 - > Amazon are really really keen on. 894 - > We want to make sure that customers have choice in those 895 - > different types of options. 896 - > But the demo tends to be kind of the same right we're gonna 897 - > prompt we're gonna try something I need to ask a developer what 898 - > do you want to go build? 899 - > We prompt it we try it we see what happens right I think one 900 - > of the things that's been most impactful for the customer 901 - > segment is going back to what can I what as a developer or an 902 - > ops team am I overburdened by on a day-to-day basis? 903 - > What can I use AI to offload to give myself the ability to go 904 - > back and really build the feature segment that I want. 905 - > And where we've seen significant success is in more 906 - > of the classic platform DevOps domain functions that really 907 - > really require developers to go build infrastructure, build 908 - > code, or build tooling configurations or APIs that are 909 - > connecting to various things. 910 - > And being able to get specific on how do we help you move away 911 - > from all of these different consolidated developer platforms 912 - > that are being used sometimes in a different way even though 913 - > it's the same branded logo in you know a 15 person team that 914 - > might be a 50,000 developer organization. 915 - > Really focusing on the areas that we're not just going to 916 - > focus on helping you build and write logic faster but we're 917 - > gonna build all of that scaffolding stuff that actually 918 - > allows that to get into production being able to show 919 - > how AI helps with that process getting it out is actually the 920 - > most critical area where we're seeing teams want to go latch 921 - > in. 922 - >

Speaker 7: Do you think that there are specific areas in in 923 - > in that you're saying that it's speeding up the SDLC basically 924 - > but do you think there's specific areas that are that are 925 - > are are being hurt yes by development? 926 - > Like if we're like Codasys right we're going faster with 927 - > codes is there other areas in the SDLC that were being hurt? 928 - >

Speaker 1: So that's actually right at the heart of what I'm 929 - > saying, right? 930 - > So what's been amazing is the explosion of different engines 931 - > that developers are using. 932 - > Because they're doing it on their personal time or they're 933 - > actually going to the enterprise and they're bringing in one of 934 - > these agentic IDEs or they're actually using or building their 935 - > own developer agent. 936 - > We've seen an explosion in code come off of those engines. 937 - > I can go faster I can build more logic against those 938 - > features. 939 - > But in the business that I support in global financials 940 - > there's still a lot of regulatory risk governance 941 - > that's required to ensure that what's being created can marry 942 - > up to the actual compliance governance regime that I need 943 - > from that feature set. 944 - > And we go into a lot of discussions with customers where 945 - > they're going well I had a hard time keeping up with all this 946 - > stuff before these AI tools were actually utilized right my 947 - > release cadence was six months, 12 months in certain capacities, 948 - > right? 949 - > So where we're seeing a really really large set of interests 950 - > beyond just offloading or using AI to less lessen the burden for 951 - > developers of all the ops things they needed to do, we're 952 - > really really working with clients and it's a why the 953 - > partnership with Horace has been so strong helping them get 954 - > better at the domain of okay I've used AI, how do I ensure 955 - > that what that AI and the human built is able to actually make 956 - > its way into production. 957 - > And when we look at how we instrument all the ways that 958 - > that comes out of the developers kind of IDE into a CI process 959 - > and be able to move that through ensuring that we're enforcing 960 - > policy are we actually managing the supply chain of what is in 961 - > that release it's been an absolutely critical aspect for 962 - > customers and it's happening really fast. 963 - > Meaning they're using all these technologies but they're coming 964 - > back to the table going I need to get better at actually 965 - > getting this out and I don't know how to make sure that my 966 - > audit team, my governance team knows that hey we built this 967 - > quickly but it actually does marry up the spec and that's 968 - > actually where harness has been so important for our clients. 969 - >

Speaker 7: So really adding AI in other areas besides just 970 - > coding but also um I I have a fear right I I feel like we're 971 - > creating this big wave or a potential tsunami of um code 972 - > that the developers are not so familiar with and it's going to 973 - > cause a bunch of like problems in the back end once production 974 - > happens, right? 975 - >

Speaker 1: Yeah. 976 - > Well it's a classic wave that we're seeing everyone's super 977 - > excited about this hype of what it can do, right? 978 - > But we are starting to really hear from clients on I love what 979 - > this can do but I need to make sure that what it can do we can 980 - > protect the actual logic are you seeing AI in um like you know 981 - > financial institutions are regular regulation and um really 982 - > you know heavily scrutinized are you seeing other uh 983 - > solutions that are helping with that piece of it meaning the 984 - > actual regulations around enforcing governance and legal 985 - > and like all of those things like because because to your 986 - > point it's like we build all this code and then we have to 987 - > regulate it. 988 - >

Speaker 7: Yeah. 989 - > Is there any is there any wave on kind of streamlining that 990 - > process? 991 - >

Speaker 1: Yeah well so we've seen that so we've seen it 992 - > actually I mean if we really go high level we've really been 993 - > working with a lot of different governing bodies in the banks to 994 - > like help them understand what secure coding actually means 995 - > what it means to actually build a supply chain of evidence for 996 - > an auditor that says this workload that you're deploying 997 - > on top of the the AWS hypervisor can be blessed because we've 998 - > built to spec, right? 999 - > I think the thing that's interesting about the question 1000 - > is we're absolutely seeing the hype wave, right? 1001 - > But it actually has driven some significant value so I don't 1002 - > want to call it a hype wave, right? 1003 - > It's a value wave. 1004 - > But what you are describing is kind of what I was describing 1005 - > previously there's so much new logic being created I need a way 1006 - > to make sure and maintain that we can actually keep up it's 1007 - > actually a paradigm we saw if we go back to the example of 1008 - > before these things were introduced, these AI 1009 - > capabilities were introduced we had a lot of best of breed 1010 - > tooling environments that were built for developers. 1011 - > When we have that happen the way that we need to build the 1012 - > body of evidence of the supply chain off of how those 1013 - > developers operate on a day-to-day basis is a very heavy 1014 - > lift, right? 1015 - > Being able to extract the information and then apply AI to 1016 - > either enforce policy or help those developers follow the 1017 - > standard practices to ensure that that actual governance 1018 - > regime is absolutely where we are seeing not only partners 1019 - > like yourselves but areas where you're taking your platform 1020 - > forward with AI to actually build capabilities natively to 1021 - > help streamline that process. 1022 - >

Speaker 7: Yeah absolutely so okay get real with me here what 1023 - > is something in the tech world business that's just a pure pet 1024 - > peeve of yours like you just can't stand it. 1025 - >

Speaker 1: Well you know to be honest with you tech world 1026 - > business I think that if I go back to the hype wave right 1027 - > there's a lot of just uncertainty of where this all 1028 - > goes and I'm watching clients like change overnight the 1029 - > perspective because the new tool came out. 1030 - > Right. 1031 - > And developers love doing that. 1032 - > We've been doing that forever right but this one's happening 1033 - > faster than I've seen in a while. 1034 - > What's cool though is so it drives me nuts when it's like 1035 - > the next day some new company comes out and then all of a 1036 - > sudden everybody only wants to go focus on that. 1037 - > It's really important though for customers to be able to test 1038 - > and try and experiment with all those things. 1039 - > So it's a little bit of a double-edged sword to say I'm 1040 - > actually frustrated by it. 1041 - > Yeah that makes total sense. 1042 - > But I will tell you that the capabilities that we're seeing 1043 - > and how all of these different engines will be utilized to help 1044 - > get developers more productive and when I say more productive 1045 - > focused on building the best features they can not 1046 - > infrastructure logic not APIs for dueling not configuration 1047 - > but really how are we going to delight customers? 1048 - > That's where this whole thing is going to go and it does 1049 - > require a lot of experimentation. 1050 - >

Speaker 7: Yeah. 1051 - > Who do you think owns the who do you think owns the should it 1052 - > be the platform team or the products team or a combination 1053 - > for AI in general? 1054 - > Like who do you think should own that? 1055 - >

Speaker 1: Because to your point governance and compliance and 1056 - > like all those things is that a platform team thing and then 1057 - > they're pushing it out to the product teams or what do you 1058 - > what is your create the platform I always I I love these these 1059 - > different team things that happen because we went from like 1060 - > development tooling team to DevOps team to cloud center of 1061 - > excellence to platform engineering. 1062 - > Now we're trying to drive an AI strategy across all of that. 1063 - > I think what you're gonna start to see is there's a lot of 1064 - > capability that if you're if you don't understand how to code or 1065 - > don't have a computer science background right that these 1066 - > different capabilities that AI is bringing to the table really 1067 - > democratize the barrier of entry for people to actually go and 1068 - > move into that space. 1069 - > And I think that what will start to happen is there will be 1070 - > a governing body of AI across all of the different use cases, 1071 - > right? 1072 - > Because it's not just development. 1073 - > We start looking at even if you just go into like basic 1074 - > financial discipline right if we're thinking about banking 1075 - > capital markets insurance the different roles that exist 1076 - > there's a lot of automation that can be created that you're 1077 - > gonna see a lot more line of business people, like people 1078 - > that are not developers and they're already doing this, 1079 - > starting to go out of building those apps. 1080 - > And what's starting to happen now is we're hearing I need to 1081 - > build more of a factory approach to how, just like we did with 1082 - > like the development teams we need to start thinking about how 1083 - > do we extend this to think of everybody as a software 1084 - > developer. 1085 - > Right. 1086 - > Right? 1087 - > And be able to actually govern how that actually operates and 1088 - > that's where the critical aspects of how are we building 1089 - > the pipeline that's pushing those things out into the actual 1090 - > production environments that are delighting customers. 1091 - > How are we ensuring we have governance in the fold across 1092 - > that pipeline right? 1093 - > So I'm seeing that start to kind of grow out where we're 1094 - > almost seeing like platform engineering, DevOps cloud center 1095 - > of excellence starting to coalesce and then a governing 1096 - > body from an AI perspective is starting to become central to 1097 - > how they're thinking about options yeah that's 100% true 1098 - > we're gonna do a segment called Ship It or Skip It okay 1099 - > non-compliant AI tools. 1100 - >

Speaker 7: I've used GPT for work I bet you have what is it 1101 - > at a ship or skip? 1102 - >

Speaker 1: You know what I think all of them have their merits 1103 - > and values I'm saying ship. 1104 - >

Speaker 7: I'm saying ship I like it all right ship or skip 1105 - > LeBron James Skip. 1106 - > Speaker: I'm a Michael Jordan guy all right one more full day 1107 - > zoom meetings in the office full day so I'm on zoom meetings all 1108 - > day on zoom games in in the office makes me want to lose my 1109 - > mind. 1110 - > Ship or skip skip 100% all right buddy thank you man thank 1111 - > you thank you for coming out to reInvent man what a crazy show 1112 - > absolutely wild it's so busy here. 1113 - >

Speaker 7: Oh it's amazing the energy is always crazy it's 1114 - > crazy yeah I appreciate you coming out look um I wanted to 1115 - > have you on to kind of get your perspective uh the tech industry 1116 - > is kind of a a crazy place right now uh AI is really taking 1117 - > it by storm and uh along with the rest of the world and I want 1118 - > to get your perspective on kind of just understanding like 1119 - > what's your what are you seeing and what's your honest opinion 1120 - > like I would prefer not marketing speak but more of a um 1121 - > yes you know hey nobody's listening right okay um but 1122 - > first is are what what are you seeing trend wise that's hype 1123 - > versus actually helpful when it comes to AI in tech? 1124 - >

Speaker 5: Okay so hype hype versus helpful um I I think like 1125 - > uh a lot of a lot of people right now are are trying to make 1126 - > claims that you can um you can you can go and uh adopt some of 1127 - > this new AI tooling especially in the delivery life cycle and 1128 - > you know 10x your throughput um I do think that there's a lot of 1129 - > potential to like you know hit uh hit velocity gains in 1130 - > particular like you know in in teams uh by by uh adopting new 1131 - > tools and methods at the same time um but I think I think 1132 - > there's been some overpromising and some like you know uh what 1133 - > feels like you know maybe uh under underproducing like you 1134 - > know in that on that account. 1135 - > I think a healthy dose of like sort of clear-eyed optimism 1136 - > around like what it really takes for for teams to actually fully 1137 - > sort of adopt tooling yeah adapt their processes right 1138 - > actually shift in from a uh talented people standpoint uh to 1139 - > to actually get in and and uh make the most use of that you 1140 - > know is the thing that we all need to be on on a trajectory 1141 - > you know to get to before we hit 10x so I think 10x is a bit of 1142 - > a hype um but but there's there's real value and 1143 - > progression there. 1144 - >

Speaker 7: Yeah I wonder that because every booth I see right 1145 - > there's AI in every booth. 1146 - > Oh yeah 100% this is basically an AI show right exactly and so 1147 - > but that's the thing it's like what what what what what's real 1148 - > what's not right yeah so what do you think is one of the most 1149 - > expensive habits um that the enterprise has that does not 1150 - > ship product. 1151 - >

Speaker 5: Oh yeah um so particularly because we're 1152 - > talking about enterprises um it seems maybe sort of crazy to say 1153 - > it in 2025 but um I think a lot of the the practices I see kind 1154 - > of stem from um a lack of really moving from a waterfall 1155 - > traditional waterfall mindset you know into uh really an agile 1156 - > you know DevOps and sort of MVP mindset. 1157 - > Yeah um I I have a lot of appreciation for where that that 1158 - > comes from you know history as well as in a lot of cases you 1159 - > know um uh budgeting and you know a financial planning you 1160 - > know standpoint like you need to understand like what an entire 1161 - > solution is going to be but I think what that often translates 1162 - > to is some of the same problems with waterfall that we we saw 1163 - > where what you start out with doesn't turn out to be you know 1164 - > what you need uh and then projects missing uh you know 1165 - > timelines uh scope increasing longer running you know cycles 1166 - > and development cycles and at the end of the day not actually 1167 - > getting product out to like your end users to uh to test out 1168 - > hypotheses um actually unlock value uh and and keep teams 1169 - > moving um and we're seeing those right in the in the the bigger 1170 - > organizations that you know are slow to move right like maybe 1171 - > healthcare and pharmaceutical these companies that have been 1172 - > around a long time um I wonder if I wonder like you know how is 1173 - > AI I mean how is AI gonna really impact that it it right 1174 - > yeah so that's the really interesting thing right um you 1175 - > know one thing I've actually been I find myself walking 1176 - > around saying a bit is that everybody's feeling the 1177 - > imperative right now around AI, right? 1178 - > But you can't you can't put AI on top of Bad, right? 1179 - > Bad processes, uh, bad technology in some cases, right? 1180 - > Um, the paradigm shift with AI in software delivery um makes it 1181 - > possible to actually change your processes very 1182 - > dramatically. 1183 - > Um but that takes like you know a real evolution, uh which 1184 - > means a lot in a lot of cases breaking down some silos that 1185 - > exist, like in many cases, like in the in the enterprise, to 1186 - > sort of lead to that waterfall process around like design, you 1187 - > know, develop, um, test. 1188 - > Um and I think uh an awareness and a willingness to kind of 1189 - > break all that down um to actually unlock the potential of 1190 - > the new technology paradigm shift is uh is what it takes. 1191 - >

Speaker 7: Yeah, I agree with you. 1192 - > Is it do you know of an AI initiative that actually 1193 - > returned cash and then one that was just really nonsense? 1194 - >

Speaker 5: Oh yeah. 1195 - > Um so I'll start out on the on the nonsense. 1196 - > Okay. 1197 - > Well, actually, you know, so where where my mind goes really, 1198 - > I I kind of think there's there's two buckets of sort of 1199 - > nonsense, like you know, things that that we've seen, again, as 1200 - > like people have felt the AI imperative around use cases. 1201 - > The first is ideas that are just frankly not AI ideas, 1202 - > right? 1203 - > Or it's like, hey, what if I could do this? 1204 - > Uh I remember um having a conversation with uh with one 1205 - > person around you know how they could basically improve 1206 - > usability you know in their application, like using AI. 1207 - > And when we really talked about what was at the heart of like 1208 - > those challenges, we realized what they really needed to do 1209 - > was make some primarily CSS changes, like you know, to make 1210 - > the system work a little bit better. 1211 - > You know, and and that comes from like trying to kind of hit 1212 - > you know whatever nail, like you know, with an AI hammer, uh, 1213 - > instead of just thinking like what am I really trying to do 1214 - > here and what's the problem. 1215 - > Um, the next class is really like areas where people just 1216 - > aren't thinking through you know, full sort of adoption, you 1217 - > know, what it's really gonna take to build you know a 1218 - > reliable um you know AI solution that people will actually 1219 - > trust. 1220 - > Uh so bad data is like usually one of is often one of those. 1221 - > Like one of the first use cases people go to is like uh 1222 - > everybody's looking up information across a lot of 1223 - > documents. 1224 - > How do I get those into a chat interface so people can quickly 1225 - > find the answer to questions? 1226 - > Um worked with somebody who was doing that at one point, put 1227 - > documents in, uh we tried it out, we realized those documents 1228 - > were not consistent. 1229 - > So it turns out you're not gonna get consistent answers 1230 - > coming out of that. 1231 - > Um so I've seen those things kind of fall down a bit. 1232 - >

Speaker 7: You know, that actually leads, like I think 1233 - > about that, and and when we talk about like shelfware, right? 1234 - > Like it's like these projects, we take on these projects and 1235 - > then they buy a solution for it, right? 1236 - > Yeah, and then for whatever reason, you know, it doesn't 1237 - > work. 1238 - > But I wonder if that's like a lack of understanding how the 1239 - > solution works and like clinging to those old ways, right? 1240 - > Or actually driving change in an organization, right? 1241 - >

Speaker 5: Yeah, exactly. 1242 - > I mean it's I really think it's both. 1243 - > Like you um you you need to have a solution that in fact, 1244 - > like you know, you thought out and not gone beyond just that 1245 - > happy pattern where I think a lot of a lot of people get stuck 1246 - > is they try something, they say, hey, it worked, but they 1247 - > haven't thought through all the different like you know, like 1248 - > edge cases. 1249 - > And once they start getting into what it'll actually take to 1250 - > make uh a solution you know fully work in a rounded out way 1251 - > that will actually be deployable, um, you know, they 1252 - > they get stuck. 1253 - > Um and then people don't they don't see the pull through you 1254 - > know in in adoption unless uh unless that trust is there that 1255 - > the solution is really gonna work. 1256 - > And then you have to also have a uh a real you know meaningful 1257 - > uh pull through like you know change management campaign, 1258 - > right? 1259 - >

Speaker 7: Yeah, absolutely. 1260 - > What do you think execs overbuy on when they're saying scale 1261 - > engineering? 1262 - > And what do you think they underinvest in? 1263 - >

Speaker 5: Yeah, uh well, I mean, classically, uh I would 1264 - > just say engineering. 1265 - > Um so when you say scale engineering, the thing that the 1266 - > thing that I've I've had plenty of conversations of like, hey, I 1267 - > need 20 engineers, right? 1268 - > Um, you know, and and a lot of the times um you know, we what 1269 - > we see is that you know specialization across different 1270 - > disciplines and and cross-functional teams, like you 1271 - > know, really lead to you know the best outcomes. 1272 - > So a lot of those disciplines and specialties, you know, that 1273 - > really support the software engineers, and I say that as a 1274 - > leader of a software engineering team, um, are uh are often 1275 - > overlooked. 1276 - > So quality engineering, uh platform engineering and devops 1277 - > that's needed to really like help the teams uh operate 1278 - > effectively, um, and all those sort of surrounding disciplines. 1279 - > So things that we often we often see aren't aren't 1280 - > initially asked for, uh, and uh and that we're we're always 1281 - > thinking about how we find the right balance of uh to try to 1282 - > really fine-tune the overall um overall team. 1283 - >

Speaker 7: Do you think right now you're seeing a trend where 1284 - > the execs are saying like let's throw AI at it? 1285 - > Maybe that, right? 1286 - >

Speaker 5: Like well, there's there's there's also definitely 1287 - > that. 1288 - > I mean, that that's the the top-down push of you know, like 1289 - > um let's let's use AI, and I'm expecting to see like you know, 1290 - > like X like you know come out of it. 1291 - > Um probably because they came to the show and saw that 100% 1292 - > 10X use. 1293 - > And everybody, and everybody, everybody on a team right now 1294 - > needs to figure out, you know, the challenge is really how to 1295 - > how to consume that, how to metabolize like you know, the 1296 - > the request of say, I need I need to be using AI or I'm gonna 1297 - > adopt I'm gonna suddenly uh have this tool available to me 1298 - > and really figure out how to go make that possible using um you 1299 - > know some some new techniques and really like like it all 1300 - > comes back to that people process technology you know 1301 - > triangle, right? 1302 - > And the tech if those if those are three sides of the triangle, 1303 - > that technology side just changed drastically, right? 1304 - > It's the people in the process that are still trying to figure 1305 - > out how to adapt to that. 1306 - >

Speaker 7: Yeah, that's very true. 1307 - > So, what do you think is a metric that we should remove 1308 - > from QBRs and maybe one that actually tells the truth? 1309 - > Oh yeah. 1310 - >

Speaker 5: Um gosh, well, so but talking about QBRs 1311 - > specifically, maybe not even one specific metric, but I think a 1312 - > hot take, I would remove agile project metrics from QBRs, 1313 - > right? 1314 - > You're the second person that said there's something there. 1315 - > I mean, whether it's it's defect counts or story points, 1316 - > those are all helpful, important like um project metrics that 1317 - > teams should use you know internally, you know, and like 1318 - > you know, with steering committees, like you know, to 1319 - > really understand what's happening, what's the help of a 1320 - > delivery. 1321 - > Um, but uh those need to be contextualized, you know, and 1322 - > they're different really for every project team. 1323 - > You know, story points are relative, they're not even like 1324 - > you know, meaningful like real measurements. 1325 - > Um you know, defect counts, like you know, like glide 1326 - > various different you know, testing strategies and 1327 - > approaches. 1328 - > A lot of context. 1329 - > There's so much, right? 1330 - > So at the end of the day, you're you're you're looking in 1331 - > a QBR setting, you're looking a little too close at that point. 1332 - > Uh but what you do you should be looking at is you know like 1333 - > how we did compared to our initial timeline estimates, um, 1334 - > and how we managed uh scope changes along the way, you know, 1335 - > and how that led to uh how that led to actual outcomes being 1336 - > there. 1337 - >

Speaker 7: Yeah, okay, great. 1338 - > Um, what's one org change that you can buy and it pays you back 1339 - > in 90 days? 1340 - >

Speaker 5: So probably the biggest thing, well, first of 1341 - > all, you know, I want to I think that siloed organization, like 1342 - > this still exists, like is is one thing, right? 1343 - > If you could org change out some of your like, you know, 1344 - > your separate teams for like those different cross-functional 1345 - > capabilities and and really get everybody working together in a 1346 - > team, I think that kind of solves what I was talking about 1347 - > earlier. 1348 - > But the the biggest thing I think I would hit on right now 1349 - > is the the AI engineering COE. 1350 - > So creating a uh a team you know that is um is also 1351 - > representative of you know your information security team uh to 1352 - > inform and adapt your policies, uh to uh establish responsible 1353 - > usage guidelines for teams, um, and then and then actually do 1354 - > enable net for teams uh broadly across an organization. 1355 - >

Speaker 7: Yeah, I couldn't agree with that more. 1356 - > How many times has somebody bought a piece of software or 1357 - > something and then just it doesn't ever get used? 1358 - > Yeah. 1359 - > Not because it doesn't work, right, but because they actually 1360 - > don't know how. 1361 - >

Speaker 5: And we've we've seen um we've we've seen a ton of 1362 - > that right now. 1363 - > A lot of people will pick it up, they try it, uh it doesn't 1364 - > immediately work, they they discard it, they say, like, you 1365 - > know, like my job's safe, I don't need to use this. 1366 - > Um and uh and then and then they uh they they move on. 1367 - > Whereas like we've been uh seeing a lot of success with 1368 - > certain sort of immersion workshops. 1369 - > Um you know, having having people spend like several days 1370 - > just learning the techniques to actually get the most from those 1371 - > tools, and then walking away and actually feeling completely 1372 - > changed. 1373 - > Like, wow, not only like this is amazing, it's gonna help me 1374 - > do work so much better, uh, and really just being excited about 1375 - > that. 1376 - > So there's some of that fear you know associated, like you 1377 - > know, that I think those COEs can really help to go go get 1378 - > people passed and realize that uh this is all really helpful, 1379 - > amazing technology should help us all do do more things. 1380 - >

Speaker 7: So if you capped headcount for a year, yeah, um 1381 - > what practice would still raise velocity? 1382 - >

Speaker 5: So um the biggest thing, and it kind of falls to 1383 - > that COE, um, I would say right now, and we've been talking 1384 - > about it here at reInvent a bunch, is uh is context 1385 - > engineering. 1386 - > Oh, interesting. 1387 - > So uh teams that that appreciate that to get the most 1388 - > the way I've been talking about it is that every time you make a 1389 - > call to a LLM completion API or uh or an agent in your IDE, um 1390 - > you might as well be pulling somebody brand new off the 1391 - > street that doesn't know anything about what you're uh 1392 - > you're trying to do. 1393 - > Um and when we do actually pull people in off the street, we 1394 - > bring them up, you know, onboard them to our organizations, let 1395 - > them know what we're trying to do, and we give them effectively 1396 - > contacts that they can go use to go do their job, make 1397 - > decisions along the way. 1398 - > Um when we when we ask an AI, an agent to go build a feature 1399 - > that like does something in an application, we need to do the 1400 - > same things, right? 1401 - > And that um that kind of builds on the concepts of prompt 1402 - > engineering um by establishing layers of context that that are 1403 - > basically that need to be maintained, you know, along with 1404 - > um you know a code base, you know, as a product is evolving. 1405 - > And that's a new sort of discipline that we're almost 1406 - > seeing emerge of really kind of how that's done and uh a hat 1407 - > that needs to be worn you know by uh by by maybe of one person 1408 - > andor um multiple people on a team to kind of take collective 1409 - > ownership. 1410 - >

Speaker 7: That's interesting because you're right. 1411 - > If if you are prompt engineering, it kind of is 1412 - > adding context to the specific code that you're writing, 1413 - > whereas, you know, yeah, that's that's actually great. 1414 - > I love that. 1415 - > Um non-compliant AI tools. 1416 - > So I don't know about you, but I uh have used Chat GPT at work, 1417 - > and it may not be compliant. 1418 - > Do we ship it or skip it? 1419 - > Oh yeah. 1420 - > 100% enterprise license. 1421 - > Yeah. 1422 - > That's the right answer for it. 1423 - > I'm still getting used. 1424 - > Um four-day work weeks. 1425 - > Uh skip it. 1426 - > Skip it! 1427 - >

Speaker 5: Sorry everybody, but we can do we can do more, we can 1428 - > do more with all these tools. 1429 - > Let's do it. 1430 - > Let's do three-day work weeks. 1431 - > Um AI code reviews. 1432 - > Uh you mean using your AI to do it. 1433 - > Uh yeah, let's let's ship it. 1434 - >

Speaker 7: Ship it. 1435 - > For sure. 1436 - > 100%. 1437 - > All right, man. 1438 - > Hey, I love it. 1439 - > I appreciate you coming in. 1440 - > Yeah, great to see you at the show. 1441 - > And uh, yeah, thanks for having me. 1442 - > All right, buddy. 1443 - > Sweet. 1444 - > Thank you for coming to reInvent, man. 1445 - > It's absolutely crazy here. 1446 - > Yeah I I really have never seen a show this crowded before. 1447 - >

Speaker 2: Absolutely. 1448 - >

Speaker 7: It's like a sporting event. 1449 - > Yeah. 1450 - > Um, I wanted to have you on. 1451 - > You're you're an engineering leader and and and you're in the 1452 - > in the midst of all this hectic time right now with AI, and uh 1453 - > the acceleration is just really quite crazy. 1454 - > Um I kind of wanted to understand from you, like, if we 1455 - > talk about um, you know, AI in general, like, where are you as 1456 - > an engineering leader kind of seeing some hype versus just 1457 - > actually productivity and efficiency tools that you get 1458 - > from AI in general? 1459 - >

Speaker 2: Yeah, so that's a great question. 1460 - > I think AI is here to help us in every step of the way in the 1461 - > whole software development lifecycle. 1462 - > I think for me, the biggest uh areas that AI has been able to 1463 - > help us is right from uh starting with the requirements, 1464 - > translation into the actual actionable task to writing code 1465 - > and to test and to deploy. 1466 - > So I personally think that AI's role is equal in every step of 1467 - > the way in the whole software development lifestyle. 1468 - >

Speaker 7: Is there any areas in that that you feel like it's 1469 - > not helping so much or we haven't quite figured it out 1470 - > yet? 1471 - >

Speaker 2: I think testing is one area. 1472 - >

Speaker 7: Testing? 1473 - >

Speaker 2: Yeah. 1474 - > Um and that's purely because uh there are gaps that I've found 1475 - > in terms of uh it being able to understand the requirements 1476 - > properly. 1477 - > And if the requirement understanding is wrong at the 1478 - > first place, it might be possible that it may come up 1479 - > with a wrong set of tests or incomplete set of tests, and 1480 - > that's probably one of the gaps that I've seen. 1481 - >

Speaker 7: Gotcha. 1482 - > Yeah. 1483 - > So what about let's say a modernization that you that 1484 - > you've gone through that worked, but uh you know, even though 1485 - > maybe you would or wouldn't repeat it again. 1486 - >

Speaker 2: Yeah, so um back in my um previous role at uh crypto 1487 - > startup, uh we um went ahead with modernizing or 1488 - > decentralizing, as they say, um the entire infrastructure, and 1489 - > we spun up different AWS accounts, different EKS clusters 1490 - > that were all self-managed, and uh we deployed all of our 1491 - > applications, microservices on those clusters, which turned out 1492 - > to be uh working really well. 1493 - > Um but I I wouldn't repeat that because some of our 1494 - > applications were super lightweight and uh we could have 1495 - > gone with like serverless technologies with lesser 1496 - > operational overhead, for example AWS Lambda or uh 1497 - > Fargate, even EKS on Fargate. 1498 - > So some of those things I I think that we can we could have 1499 - > uh done much, uh we could we could have gone much simpler 1500 - > route. 1501 - >

Speaker 7: So if you cut your platform um spin by 30%, um what 1502 - > goes first and what gets the stake? 1503 - >

Speaker 2: Yeah, um so again there are nice to have things in 1504 - > every platform team and there are must-have must-have things. 1505 - > Um for nice to have things, things like uh duplicate vendors 1506 - > uh that offer pretty much redundant capabilities or 1507 - > overlapping capabilities. 1508 - > Um there are low-impact environments, uh, there are uh 1509 - > redundant tools, complex tools uh that are no longer relevant 1510 - > or no longer used. 1511 - > But my favorite is the unused and underused resources. 1512 - > And I've been able to reduce cost by more than 40-50% in my 1513 - > previous roles, but just focusing on cutting down or 1514 - > shutting down all the underused and underused resources and also 1515 - > right sizing of some of those resources. 1516 - > So I think for me that would be one area, uh, and what must 1517 - > have is CICD and observability. 1518 - > I think observability, personally speaking, there's no 1519 - > limit to it. 1520 - > You can always have more dashboards, more insights, more 1521 - > information about your platform. 1522 - > Some of those things you would not even imagine or know that 1523 - > you would need those, but I think there are always some 1524 - > scope and CICD equally. 1525 - >

Speaker 7: That's a great point. 1526 - > Why do you think dashboards go unused? 1527 - >

Speaker 2: Because they are statically created. 1528 - > Um I have use case one, two, three, and I create widgets for 1529 - > those one, two, three. 1530 - > Or I create dashboards for those one, but there's always 1531 - > 1.5 or 2.5 that are some corner cases that we always miss. 1532 - > And that's why with the um with the AI capabilities, what we 1533 - > are trying to do right now is have agentic solutions built to 1534 - > provide us all of the material, all of the information. 1535 - > Um, and that's really uh uh becoming a conversational way of 1536 - > getting to know information that you're looking for. 1537 - > Instead of statically defining dashboards, you just simply ask 1538 - > questions with your platforms, with your observability tools 1539 - > that can answer you the questions that you're looking 1540 - > for by uh you know going behind the scenes and understanding and 1541 - > uh analyzing all of the data sets and metrics and logs and 1542 - > events and so on. 1543 - > So I think that's the direction we need to go towards. 1544 - >

Speaker 7: Why do you think so? 1545 - > This is something I I wonder quite in depth is if you look at 1546 - > an organization and they have the finance department and the 1547 - > marketing department, the sales department, those departments 1548 - > run heavily on metrics. 1549 - > They live and die by metrics. 1550 - > Why do you think the engineering group has not been 1551 - > able to adopt that? 1552 - > Because I work with a lot of engineering organizations and um 1553 - > like adopting or getting Dora metrics just in general, it 1554 - > seems to be very difficult. 1555 - > And when we do, they oftentimes they don't want to be measured. 1556 - > I'm just curious, like what like why do you think that is? 1557 - >

Speaker 2: I think it comes from the fact that a lot of 1558 - > engineering organizations are very top-down focused. 1559 - > So they're given the requirements, given the you 1560 - > know, use cases that they want, given the business problems that 1561 - > they want to, or they're asked to build the solutions, and they 1562 - > kind of get narrow-sighted to achieve those results and 1563 - > provide those those outcomes. 1564 - > And I think that's that's one uh approach or mindset that's 1565 - > where we miss out on a lot of results or matrix-driven 1566 - > approach. 1567 - > Um, I've seen all I've also seen some organizations working 1568 - > really bottoms-up, and that's where we define really the 1569 - > ground level matrix, and we go by that. 1570 - > We try to achieve that right from the get-go. 1571 - > Uh, even when the requirements are not very clear, uh, we we 1572 - > define the results. 1573 - > For example, um, in one of my previous organizations, when I 1574 - > was trying to build this platform as a product, I defined 1575 - > some of the core metrics. 1576 - > Like we want to reduce the cost by 30%, we want to increase the 1577 - > utilization by 50%, um, we want to achieve the availability SLA 1578 - > by 90%. 1579 - > So some of these metrics, once we defined, then we started 1580 - > building the products around those metrics, and that's where 1581 - > we were able to always backtrack and check how well we are 1582 - > progressing and what are the gaps that we need to fill to 1583 - > achieve those results. 1584 - >

Speaker 7: So, does it get a little bit of tunnel vision with 1585 - > metrics like again, based on initiative? 1586 - > It's kind of like the metric becomes tunnel vision, and it's 1587 - > like, okay, we have to hit this one thing, and then maybe other 1588 - > stuff goes by the wayside, right? 1589 - >

Speaker 2: Yeah. 1590 - >

Speaker 7: Yeah. 1591 - > So um what's something in engineering that you just 1592 - > absolutely think is a waste of time and we should stop doing 1593 - > it? 1594 - >

Speaker 2: It might be. 1595 - > You can be honest. 1596 - > Yeah. 1597 - > Like I said, I was going to say that I might be controversially 1598 - > saying this, but uh I think uh the process-driven software 1599 - > development, um quote-unquote agile uh methodologies, um they 1600 - > are oftentimes over-engineered. 1601 - > Um there are practices like predicting the capacity, 1602 - > bandwidth of the team, and uh but my view is that engineering 1603 - > is unpredictable. 1604 - > That's the fun part of it, right? 1605 - > If it's so much predictable, then it's probably not, it 1606 - > should not shouldn't be called engineering, it should be called 1607 - > something else. 1608 - >

Speaker 7: Um so if you're not running into problems and and 1609 - > having to iterate, yeah, then it's probably you're probably 1610 - > not doing the right thing. 1611 - >

Speaker 2: Yeah. 1612 - > For example, um back in the days when I was building a 1613 - > platform team at T-Mobile, um I achieved like 90 story points in 1614 - > one sprint and 10 story points in another sprint. 1615 - > And I loved it because the unpredictable part was the fun 1616 - > part for me. 1617 - > And I think oftentimes I've seen that uh in the pursuit of 1618 - > following those strict guidelines and processes, we 1619 - > kind of lose the sight of what we really need to deliver and 1620 - > how should we innovate? 1621 - > I think that's the part we should have. 1622 - >

Speaker 7: Do you think then that measuring work delivered, 1623 - > or like like I guess how would you quantify that? 1624 - > Like, how would we say what good is if you can do 90 story 1625 - > points one week and 10 the next, right? 1626 - > Like how what is the what is the end like result that we 1627 - > could measure and say, oh, regardless of number of story 1628 - > points, we this X was accomplished. 1629 - >

Speaker 2: Yeah, I think that it should be measured in terms of 1630 - > the user impact. 1631 - > Um what that end user. 1632 - > End user impact. 1633 - > End user. 1634 - >

Speaker 7: Okay. 1635 - >

Speaker 2: Or even if the end user is your own team, uh let's 1636 - > say I'm building an internal tool for my own team. 1637 - > What was the impact? 1638 - > Yeah, how many users actually try to use it, how many times it 1639 - > was used. 1640 - > Those are the kind of things you should always measure. 1641 - >

Speaker 7: If AI disappeared tomorrow, um what's that boring 1642 - > platform investment that sticks around and is still always gonna 1643 - > work? 1644 - >

Speaker 2: Yeah, I'll go back to my previous answer CI CD and 1645 - > observability. 1646 - > CI CD. 1647 - > Observability and CI CD are the two things. 1648 - > Gotcha. 1649 - > Very important part, right? 1650 - > Absolutely. 1651 - > You just can't um get away with that. 1652 - >

Speaker 7: What's the most unpopular policy change that you 1653 - > made that actually sped delivery? 1654 - >

Speaker 2: That actually sped delivery. 1655 - > Umpopular change. 1656 - >

Speaker 7: Yeah, unpopular change. 1657 - > Okay. 1658 - >

Speaker 2: Um We were using a lot of open source tools to the 1659 - > extent that it was almost breaking our internal monitoring 1660 - > stack by providing us a lot of false positive. 1661 - > Something changes on the upstream, downstream changes, 1662 - > are not prepared. 1663 - > So we kind of were in a continuous loop of fixing those 1664 - > problems by providing some or like adding some one-off or 1665 - > Glucode type solutions. 1666 - > And that was getting out of hands. 1667 - > So I proposed a policy of stop using open source, at least for 1668 - > those specific use cases. 1669 - > It was very unpopular as you can imagine. 1670 - > But we ended up building our own in-house monitoring stack 1671 - > with things like HealthWatch and SmokeCest Suite. 1672 - > And that kind of gave us much deeper and broader visibility 1673 - > into the health of our systems. 1674 - > And that's not true for every use case. 1675 - > I mean, the world lives on open source, so you not using open 1676 - > source is not a great idea for everything, but for certain 1677 - > things where the open source was not really maintained, was not 1678 - > well supported by the community, using those kind of tools was 1679 - > not really a good idea. 1680 - > And that kind of was the change that we made, and we kind of 1681 - > really achieved great results with that. 1682 - >

Speaker 7: What's the single AI tool that you're that you guys 1683 - > are using right now that you feel you're getting the most 1684 - > value out of? 1685 - >

Speaker 2: I think we are using GitHub Copilot and we are 1686 - > leveraging it to the full extent. 1687 - > We use it right from the requirements, translation, which 1688 - > is in most cases very ambiguous, and we translate 1689 - > those requirements into very specific user-actionable tasks, 1690 - > and from there software development to testing to 1691 - > deployment. 1692 - >

Speaker 7: Are you guys using it for document? 1693 - > Like are you using AI for documentation? 1694 - > I feel like that's a really easy one, right? 1695 - >

Speaker 2: It just helps we use we use AI for um writing our 1696 - > runbooks. 1697 - > So any troubleshooting, any um SRE DevOps type work, uh, we 1698 - > create extensive runbooks for different kinds of scenarios, 1699 - > and we use AI to do that. 1700 - >

Speaker 7: What's one vendor narrative that you think 1701 - > misleads engineering leaders today? 1702 - >

Speaker 2: Wow, that's a that's a great question. 1703 - > I think uh I've seen that build versus buy is still a grey area 1704 - > for a lot of engineering leadership. 1705 - > Um there's a misconception that vendors can abstract 1706 - > everything. 1707 - > Um, a lot of it is true, but there are areas where uh you 1708 - > have to customize the solutions, you have to build in-house 1709 - > tools to sort of augment what the vendors can offer and uh 1710 - > support. 1711 - > Um and again, there's no right or wrong answer on when to buy 1712 - > versus build. 1713 - > Uh it really depends on the use case, your skill sets, your 1714 - > bandwidth, your priorities. 1715 - > And um I use these parameters as like the variables in the 1716 - > equation uh to come up with the decision whether we should build 1717 - > or we should buy, with whether we should do both. 1718 - > Uh you know, buy first and then build on top of something that 1719 - > we bought. 1720 - >

Speaker 7: So how much do you think um the build option is 1721 - > driven by job security? 1722 - >

Speaker 2: Yeah. 1723 - > Um again, so um job security, the whole uh narrative about job 1724 - > security has like two, three different um verticals or uh 1725 - > sort of thoughts behind it. 1726 - > Uh number one is uh tribal knowledge. 1727 - > People tend to keep uh information and you know that 1728 - > gives them the job security. 1729 - > Sometimes people build unnecessarily complex systems so 1730 - > that in order to maintain, they would they would remain in the 1731 - > job. 1732 - > Uh I think all of those things are very very much present even 1733 - > in today's uh world. 1734 - > But I personally think that AI and automation and you know the 1735 - > whole narrative of buy uh is to not reduce the scope of job 1736 - > security, but it's to actually improve. 1737 - > Um and by that I mean when you buy and you you kind of 1738 - > demonstrates that it is useful and it can solve the problem 1739 - > that would have taken you months or weeks to build, and now you 1740 - > just bought and like bought a solution and you're able to do 1741 - > it much faster. 1742 - > You're also improving on your own credentials, right? 1743 - > Right. 1744 - > So so I think that's a that's a very uh overlooked point, and I 1745 - > think that that's something that people should remember when 1746 - > they think like if I buy something, I might reduce my job 1747 - > security. 1748 - > That's not the case. 1749 - >

Speaker 7: I agree. 1750 - > Um okay, ship it or skip it. 1751 - > AI code review, skip it. 1752 - > Skip it. 1753 - > Why? 1754 - > You don't trust it yet? 1755 - >

Speaker 2: Because it I'm I want to first trust the code written 1756 - > by AI. 1757 - > Okay, only then I will trust the code review by AI. 1758 - >

Speaker 7: Okay, ship it or skip it, four-day work week. 1759 - > Ship it. 1760 - > Ship it. 1761 - > I love this guy. 1762 - > Okay, ship it or skip it. 1763 - > No deploy Fridays. 1764 - >

Speaker 2: Oh, hell yeah. 1765 - > Ship it. 1766 - >

Speaker 7: Ship it. 1767 - >

Speaker 2: Yeah. 1768 - >

Speaker 7: I love it. 1769 - >

Speaker 2: Yeah. 1770 - >

Speaker 7: Awesome, brother, brother. 1771 - > Thank you for coming. 1772 - >

Speaker 2: Absolutely. 1773 - > Appreciate you. 1774 - > Great meeting with you and pleasure being here. 1775 - >

Speaker 7: Loved having you. 1776 - >

Speaker 4: Those were really insightful discussions, Thomas. 1777 - > So, for our listeners, what would be one takeaway for 2026 1778 - > uh for either their engineering excellence goal or their 1779 - > modernization? 1780 - > So you have assessed SDLC for a lot of companies. 1781 - > Uh, what would be one golden rule they can follow? 1782 - >

Speaker 7: Yeah, I would say if I had to boil it down, I would 1783 - > say do not mistake AI acceleration for engineering 1784 - > excellence. 1785 - > They are very two very different things. 1786 - > I think engineering excellence in 2026 is going to really come 1787 - > down to whether your system can absorb speed without losing 1788 - > trust. 1789 - > And the trust word there is absolutely important. 1790 - > If we think about AI agents and um implementing those, the 1791 - > number one thing that will prevent us from doing that is 1792 - > trusting that the agent will take the actions that we want it 1793 - > to and not go and do something destructive. 1794 - > Um, that means that strong platforms, embedded governance, 1795 - > uh, clear ownerships, I think those are just things that um 1796 - > are of the utmost importance. 1797 - > I've talked a lot about utilizing like an internal 1798 - > developer portal to be a context layer for your agents and 1799 - > things of that nature. 1800 - > It's it's really going to be a matter of trust and building in 1801 - > a um a process to uh make sure that your agents and um other AI 1802 - > pieces have the ability to check for governance and safety 1803 - > guardrails in order to take action. 1804 - >

Speaker 4: Yeah, I love that uh touch on guardrails, like 1805 - > guardrails not gates. 1806 - > I think uh uh harness field CTO office also repeats that uh 1807 - > that uh frame that it's guardrails not gates. 1808 - > So uh you you touched on this before. 1809 - > What would be uh one make or break factor for platform 1810 - > engineering teams within the next 12 months based on your 1811 - > re-invent chats? 1812 - >

Speaker 7: Yeah. 1813 - > I think um the one that comes to mind for me, make or break, 1814 - > in my opinion, is going to be context and operating model 1815 - > maturity. 1816 - > I think that um a lot of teams are still thinking about AI 1817 - > mostly in terms of models and tools. 1818 - > Um, but one of the most important themes that I heard uh 1819 - > was that the the future is going to be much more context 1820 - > driven. 1821 - > Uh and we know that you know anybody writes a message to GPT, 1822 - > uh the less context you give it, the the worse the response 1823 - > is going to be. 1824 - > The more context you give it, the better it's gonna be. 1825 - > And so when we translate that into engineering, um I think you 1826 - > know the teams that win over the next 12 months are gonna be 1827 - > the teams that do two things really well. 1828 - > First, they create a strong golden path through platform 1829 - > engineering where this where there's a secure and governed 1830 - > path that's also the easiest path. 1831 - > I think that's really important. 1832 - > Uh and then the second piece is invest in context. 1833 - > So, um, how do we do that meaningful information and 1834 - > standards, workflows, uh, and clarity that help both the human 1835 - > and the AI ultimately operate more effectively. 1836 - >

Speaker 4: Thomas, thank you so much for sharing these insights. 1837 - > If our listeners want to connect with you, learn more 1838 - > about your work, where can they find you online? 1839 - >

Speaker 7: Yeah, absolutely. 1840 - > So I'm on LinkedIn at Thomas DocsDator. 1841 - > Um, I uh I again, as I mentioned before, for harness, I 1842 - > do a lot of consulting work for our clients in um doing uh 1843 - > really SDLC assessments that are utilizing Dora and Accelerate 1844 - > and really looking at it from a uh not from a, hey, I'm a vendor 1845 - > and how can I get you to give a product of mine, but really 1846 - > from a perspective of saying what are the people, processes, 1847 - > and tools that you're using to um operate your SDLC and uh 1848 - > really helping uh organizations zoom out, look at their whole 1849 - > SDLC, and then if there's a great harness product that could 1850 - > help them, awesome. 1851 - > Um that's that's the that's the the direction that I've had a 1852 - > lot of great success with our with our current customers. 1853 - >

Speaker 4: So we add the links in the chat. 1854 - > That was Thomas Doc Stator, engineering excellence at 1855 - > harness. 1856 - > Thanks so much for an amazing season four. 1857 - > I'll see you all in season five.