Doing AI Is Easy. Doing It Well Is Hard.

1 - > SPEAKER_03: One of my uh engineering leaders, uh, she 2 - > says, and it's very, very true, doing AI is easy. 3 - > Doing AI well is a lot of hard work. 4 - > SPEAKER_00: Doing AI is easy. 5 - > Doing AI well is hard work.

6 - > That may be the clearest description yet for where 7 - > enterprise AI stands today. 8 - > Because the tools can already write code, build prototypes, 9 - > and compress weeks of work into just hours. 10 - > But becoming AI native requires much more than handing employees 11 - > a coding assistant. 12 - > It changes who can build, how teams organize, how quickly 13 - > ideas reach production, and how much risk an organization can 14 - > create before anyone realizes what happened.

15 - > So on today's episode of the AI Proving Ground Podcast, we'll be 16 - > talking with Raj Chopra, Cisco's Chief Product Officer for 17 - > Security, and two AI leaders here at WWT, Brian Orbald and 18 - > Joe Berger, about why when code becomes cheap, the scarce 19 - > resource is no longer the ability to produce it. 20 - > It's the judgment to decide what should be built, the domain 21 - > expertise to make it useful, and the governance to keep it all 22 - > secure. 23 - > We recorded this episode live on the show floor at Cisco Live.

24 - > And throughout the conversation, Raj, Brian, and Joe explore what 25 - > separates organizations merely using AI from those genuinely 26 - > changing how they operate, and what happens when autonomous 27 - > agents can reason, access enterprise systems, and take 28 - > action on their own. 29 - > So let's get to it. 30 - > SPEAKER_01: Oh my gosh, yeah. 31 - > I would say over the past, yeah, over the past six to 12 months, 32 - > we've just seen a huge uptick.

33 - > I think if you start looking at a lot of the different use cases 34 - > of AI that people have developing over the past few 35 - > years, they really honed in on coding assistance because you 36 - > were able to see immediate impact within your organization. 37 - > And I think we're at that point now where a lot of companies 38 - > have adopted these tools. 39 - > Now it's around how does this actually change my organization? 40 - > It's not just I have people using the tools, but how does it 41 - > change my software development lifecycle?

42 - > How do I start shipping code quicker? 43 - > How do I test for bugs faster? 44 - > I know we're going to get into security talk here in a little 45 - > bit, but it's really creating this big momentum for the 46 - > software development team. 47 - > And as well, you're even seeing it make its way into sort of 48 - > your everyday knowledge worker in terms of low-code tools and 49 - > now everyone can start developing really quickly.

50 - > So just tremendous opportunity out there that I think a lot of 51 - > organizations are, they've already been using it or they're 52 - > kicking the tires on, at least for sure. 53 - > SPEAKER_00: Yeah, and Brian, I mean, he's talking about how 54 - > it's changing not just the developer teams, it's it's 55 - > changing the whole course of an organization. 56 - > What are you seeing in terms of implications of AI native 57 - > engineering and what it means for an organization, not just 58 - > from a development standpoint, but infrastructure and policy 59 - > and things like that?

60 - > SPEAKER_02: Uh, you know, first thought that I have before 61 - > infrastructure and policy is just the impact to Joe's point 62 - > on the on the workforce. 63 - > So you start seeing the tipping point from AI native engineering 64 - > into workforce AI, and you've got the the entire workforce has 65 - > the ability to do things that were never possible for them 66 - > before. 67 - > And I think that's what ultimately has a consequence on 68 - > behalf of B infrastructure policy, the way you architect 69 - > things to Raj's point, like everything's changing.

70 - > The problems of yesterday are not the problems of tomorrow. 71 - > And so we have to solve things in a very different manner now. 72 - > And that is just like we've gone through over the last 25 years, 73 - > it's another iteration of this requires new ways of thinking 74 - > about infrastructure design. 75 - > You're you're accounting for ways in which you have to 76 - > connect systems in a different manner, which ultimately 77 - > requires how you secure them and in just a new frame of mind, new 78 - > mindset.

79 - > So I'm I'm excited about the fact that we've got this ability 80 - > to go create new experiences for our user base, but it's also 81 - > incumbent upon us to make sure that we are empowering them in a 82 - > way that's responsible, it has an ability to scale and provides 83 - > a delightful experience for them. 84 - > SPEAKER_03: Certainly a great segue into Raj. 85 - > Yeah, go ahead. 86 - > I mean, hopefully this is not a hot take anymore, but AI is 87 - > exceptionally good at doing hard things.

88 - > Right. 89 - > For a long period of time, people have this notion, which 90 - > is like, okay, we'll use AI for as an assistant or so on and so 91 - > forth. 92 - > Hopefully, again, one more time, this is not like I'm not blowing 93 - > anybody's mind off. 94 - > AI is very, very good at doing hard things.

95 - > So the real question comes: do you have people that have agency 96 - > to really challenge what are the big things to do? 97 - > Right? 98 - > If writing code is essentially free, what are the big problems 99 - > to solve? 100 - > And what we are finding from our developer experience is that 101 - > people have so many high-quality problems to solve that you 102 - > should expect not just the professional skill set, but the 103 - > cultural mindset to shift to people who have a lot of agency.

104 - > I think the distance between the end customer and the developer 105 - > is gonna shrink. 106 - > The surface areas on which people interact are going to 107 - > change. 108 - > The way uh vendors like us enable partners like you all is 109 - > going to change. 110 - > It is not gonna be the usual lecture, and here's the 111 - > document, and so on and so forth.

112 - > But every aspect when you lay it down, think of the hardest 113 - > problems. 114 - > Those are the first ones that AI is solving, and that is 115 - > culturally changing the composition of the team, not 116 - > just skill-wise. 117 - > SPEAKER_00: Okay, so the tools exist, the capability is there, 118 - > but the shift from using AI to write code to being the kind of 119 - > organization that's actually AI native, that's a cultural change 120 - > as much as a technical one.

121 - > And it doesn't happen by accident. 122 - > SPEAKER_01: Yeah, hey Raj to that point, are you noticing 123 - > because obviously, as as developers move from sort of 124 - > writing code to now being the orchestrator of stories and a 125 - > building, are you seeing that shift within Cisco? 126 - > And is it also is it kind of challenging the mindset of the 127 - > engineer of like my job yesterday was this, I now have 128 - > to go think like this, and it's different. 129 - > I mean, there's a shift happening.

130 - > SPEAKER_03: It definitely is, and I think these shifts are 131 - > those of mindsets, not of titles. 132 - > So most people now in the team should think of themselves as 133 - > member, member of technical stuff. 134 - > Yeah, right. 135 - > Yeah, it is not like you are a developer, you are an you are 136 - > not a developer, you are a QA, you are a PM, you're a designer, 137 - > what have you.

138 - > But it really is you're a member of the technical stuff. 139 - > And that technical stuff gets applied to all sorts of 140 - > different kinds of problems. 141 - > It truly is where you're thinking about how exactly 142 - > should we solve the most urgent, the biggest problems, and then 143 - > have the empowerment to go solve it because the cost of failure 144 - > of trying something and failing at it is small. 145 - > Oh, yeah.

146 - > So if the cost of failure is small, what else would you try? 147 - > I think that's the mindset. 148 - > So it's I think the org structure is evolving more 149 - > organically than you would think. 150 - > It is mission-oriented, it is those who take agency, it's 151 - > where their job becomes more about collapsing whatever the 152 - > inefficiencies are rather than being told, go do this, go do 153 - > that.

154 - > SPEAKER_01: And let me ask you, because I know Cisco's got a 155 - > very large development organization, what did you all 156 - > do to enable this? 157 - > Was it training programs? 158 - > Or is it, hey, get the tools in everyone's hand, let them try it 159 - > out? 160 - > Like how how did you get over this adoption curve?

161 - > SPEAKER_03: I think the most effective way that we did, and a 162 - > bunch of those things were tried and still are tried. 163 - > I think, first of all, making these tools available for people 164 - > to experiment and sometimes fail. 165 - > I mean, that's totally okay. 166 - > But really, availability of the tools is very, very important.

167 - > The the two big things I would say is expressing the mission of 168 - > the company in uncomplicated words. 169 - > Very, very important. 170 - > And then second is driving the lived experience of urgency. 171 - > There is no time to waste.

172 - > Again, one more time. 173 - > This is not about like foster, harder, faster. 174 - > It is more about the fact that if we are not solving the big 175 - > problems, then we are not being responsible. 176 - > Yeah.

177 - > SPEAKER_00: Brian, what do you think about that? 178 - > I mean, you're you're interfacing with clients every 179 - > day across a wide variety of industries. 180 - > Is that what many organizations are doing, or is there a 181 - > learning curve there? 182 - > SPEAKER_02: I think there's a little bit of all, right?

183 - > There's you think your experience is gonna vary. 184 - > Cisco's obviously a leading technology solutions 185 - > manufacturer provider, developer innovator. 186 - > I think organizations that don't necessarily have that kind of a 187 - > culture built into the fabric of how they operate, that that may 188 - > represent, that may manifest itself in a development 189 - > workforce that may be a little bit more hesitant to lean in and 190 - > understand that it's mission-oriented as opposed to 191 - > project-oriented or code-oriented.

192 - > And what we've seen with clients that have really, I think, made 193 - > it past that that made it around that curve is kind of forcing 194 - > the issue to have the conversation with your workforce 195 - > that it is gonna behave differently. 196 - > We're asking you to do things differently, and what you were 197 - > doing yesterday might be a little bit different tomorrow. 198 - > Your skills are still super valuable. 199 - > We still value all of what you've done for the organization 200 - > or in your career around software, but we're gonna apply 201 - > it differently now.

202 - > And so I think being forward with that in areas where maybe 203 - > the culture isn't as innovation-leaning is important, 204 - > but I think it's a great question from Joe just around 205 - > being thoughtful around the cultural shift and the adoption 206 - > curve matters deeply in this stuff because it's not just 207 - > about acquiring a tool and giving it to your workforce and 208 - > hoping great things happen. 209 - > There has to be a purpose and a plan behind it.

210 - > And I think the mission way you say the standard diraj is a 211 - > great way to frame it. 212 - > SPEAKER_03: If I may, one of the other things, Brian, is what we 213 - > internally have found is I think the showcasing the failures, 214 - > yeah, and not in a way like you're calling somebody out, but 215 - > celebrating that this was tried and this was a learning from it 216 - > is is very helpful. 217 - > And then the other part that we again live through talk rather 218 - > showing rather than talking.

219 - > Right. 220 - > So instead of a very elaborate sort of spec or this or that or 221 - > the other PRD specification, why don't you just give it a lived 222 - > experience through a prototype? 223 - > Prototype it, yeah. 224 - > Right.

225 - > And I'm telling you, like 10 out of 10 times, once people have 226 - > used the tool, they feel so much of empowerment. 227 - > Yeah. 228 - > Right. 229 - > One other additional thing, and this goes back to your your 230 - > skills, are really important, is AI is generic, right?

231 - > And it will give generic answers if you give it generic prompts 232 - > and generic everything. 233 - > What makes a product really hum and really sing is that domain 234 - > expertise that needs to come. 235 - > So it is not that a generic AI is gonna solve bisco problems, 236 - > domain expertise amplified through AI is what solves real 237 - > world problems. 238 - > So that skill set stuff is very, very important.

239 - > Code is just a means to express it. 240 - > SPEAKER_02: Yeah. 241 - > I I've I've been using kind of a framework of you've got to 242 - > create a mindset around adoption to build confidence that leads 243 - > to having, and ultimately you're gonna see change. 244 - > Yep.

245 - > And you know, it's it's being deliberate about those things. 246 - > Again, not just hoping for the best because you've provided a 247 - > tool. 248 - > We've given tools to people many, many times with mixed 249 - > results. 250 - > And I think in this case, we're being at least internal to 251 - > worldwide, but very, very deliberate.

252 - > It sounds like Cisco is as well, but you're you're focusing on 253 - > the how and the why, not just the what. 254 - > Yep, absolutely. 255 - > SPEAKER_01: I I would also add, and I think Brian can attest to 256 - > this, and Raj, I think Cisco's probably in the same boat. 257 - > This starts from the top.

258 - > Top down, 100%. 259 - > And you know, if you know anything about our CEO, Jim 260 - > Kavanaugh, I mean, he's very passionate around AI. 261 - > And so every Monday on our AI calls throughout the week, he's 262 - > attending, he's asking the questions. 263 - > But but when we've seen customers who have really dived 264 - > into AI over the past couple of years, it really is because 265 - > they've got someone at the top driving it and making sure the 266 - > success and pushing it down.

267 - > To your point around the mission, it's not just a product 268 - > set, it's really how do I change the organization with these 269 - > tools? 270 - > But that's driven from the top down. 271 - > And that's really how you can moment. 272 - > SPEAKER_02: I think sending us.

273 - > Yeah. 274 - > Because we want to see proliferation of our user base 275 - > uh really taking full advantage of these capabilities. 276 - > SPEAKER_03: One of my engineering leaders, she says, 277 - > and it's very, very true, doing AI is easy. 278 - > Doing AI well is a lot of hard work.

279 - > SPEAKER_02: Oh, yeah. 280 - > But I I I think it matters to get people off the starting 281 - > block because when they realize what's possible, then the 282 - > curiosity kicks in. 283 - > And that's when I think real change can happen, and that 284 - > unlocks all sorts of new possibilities that we weren't 285 - > really able to tap into before it weren't even in our vision 286 - > before. 287 - > SPEAKER_00: So the adoption curve is real, but it cuts both 288 - > ways.

289 - > As tools proliferate and everyone from developers to 290 - > business leaders starts building with AI, a new risk surfaces 291 - > that most organizations aren't ready for. 292 - > This episode is supported by cognition. 293 - > Cognition's AI powered developer tools, windsurf and dev and 294 - > deliver deep code-based understanding, intelligent code 295 - > completion, and the autonomous engineer capabilities to help 296 - > teams embrace the future of software engineering.

297 - > Supercharge your development team with a cognition platform. 298 - > Well, Joe, I mean, you know, Brian's talking about how you 299 - > know the unlimited possibilities once it kind of clicks for 300 - > people, but are we also kind of entering an era where you know 301 - > we've all heard of shadow AI? 302 - > Are we gonna have shadow engineering where you know 303 - > Brian's making that that's already out there, Brian? 304 - > SPEAKER_01: If he's writing code now and well, if he's prompting 305 - > for code, well, that's where kind of Brian talked a little 306 - > bit about the governance.

307 - > And you know, you hear more and more people talking about the 308 - > notion of the harness where you've got to make sure your 309 - > data sets are in place, your governance is there, you've got 310 - > security behind it because it does start spreading fast. 311 - > If you don't have these other things already in place, that's 312 - > where you're gonna get yourself into trouble. 313 - > Yeah, and so just having that team. 314 - > I know we talked a little bit about you know the COEs getting 315 - > formed in a lot of the organizations, just having that 316 - > attention on these things now before your departments go out 317 - > and just start buying stuff off the shelf.

318 - > You don't know where your data is going, having that COE and 319 - > governance now is really gonna be a tremendous asset as this 320 - > gets bigger. 321 - > SPEAKER_00: Yeah, Raj, I mean, we're talking about code being 322 - > created faster than ever and tools in everybody's hands. 323 - > What other risks do we have to be concerned about as we enter 324 - > this era where things are gonna be popping up all the time? 325 - > SPEAKER_03: Yeah, so I think the biggest risk, there are two 326 - > parts to it.

327 - > One is agents have reasoning capability. 328 - > What does that mean in normal English? 329 - > We used to write, I've written software, right? 330 - > You're writing software.

331 - > Sort of. 332 - > Is we used to write code, right? 333 - > It would be if this happens, do this. 334 - > If then else, so on and so forth, right?

335 - > So there was a codified logic. 336 - > You give the same input ten times, you're gonna get the 337 - > exact same output, right? 338 - > These agents do not have that kind of methodical thing. 339 - > It is what is called reasoning.

340 - > Depending on the input, I might take a different action. 341 - > So that is number one, the risk part. 342 - > The other thing is agents take action. 343 - > APIs don't take actions by themselves, devices don't take 344 - > actions by themselves, right?

345 - > There is, those are all in some way waiting for instruction. 346 - > But agents take action. 347 - > And in that world, there are those are the, I would say, from 348 - > a technical perspective, those are the two biggest risks. 349 - > Because between reasoning and the ability to take action, what 350 - > really becomes important is yeah, accessing that 351 - > Salesforce.

com or your Jira instance or whatever, that may 352 - > be perfectly okay, right? 353 - > For an agent. 354 - > But the intent behind that access becomes very important, a 355 - > very simplistic example. 356 - > It's one thing to say, I'm asking my agent to have access 357 - > to email so it can summarize this email that Brian has sent, 358 - > because Brian sent me very long emails.

359 - > He's excited about the software. 360 - > So it it's one thing to say, I'm gonna summarize an email. 361 - > But that same access, if you're scraping my entire mailbox, very 362 - > different intent. 363 - > So codifying that reasoning along with the ability to take 364 - > action action and putting it behind an intent, that is the 365 - > key.

366 - > Now, there is no protocol for intent, there is no spec that is 367 - > out there in the internet, but that is what we as an industry 368 - > and we as Cisco are trying very hard to distill intent into both 369 - > visibility as well or observability as well as policy 370 - > so that you can match up the intent of the agent because 371 - > these are reasoning and they take action. 372 - > So, how do you allow looking at one email to summarize? 373 - > Perfect. 374 - > Looking at my entire mailbox, same function, but calling my 375 - > entire mailbox to scrape every single email, probably not okay.

376 - > How do you see, Raj? 377 - > SPEAKER_00: Stick with you. 378 - > How do you see identity and access evolving as as AI becomes 379 - > more autonomous? 380 - > I mean, you're the way you would talk about it previously would 381 - > be for a person or for team, yeah, or for an application that 382 - > didn't have agentic reasoning.

383 - > So how is it shifting over time and where is it going? 384 - > SPEAKER_03: Yeah, so two-step process. 385 - > One is authentication. 386 - > Authentication, which is just giving an agent an identity, 387 - > just like back in the old days, right?

388 - > No, you wouldn't let somebody come in, plug a device into the 389 - > wall socket, right, whatever, and say, now you are on the 390 - > internet or you have a DHCP address. 391 - > Yeah. 392 - > Right? 393 - > We wouldn't do that.

394 - > Yeah. 395 - > We do a posture check and all that kind of stuff. 396 - > So when somebody brings an agent, before you give it an 397 - > identity, you've got to make sure that this agent is worthy 398 - > to be on your environment. 399 - > Because this agent is going to take action all around.

400 - > So how do you do that? 401 - > You basically posture check that agent, the skills that it has, 402 - > does it uh have access to MCP servers that might be corrupted, 403 - > etc., etc. 404 - > So you're basically checking the posture of that device.

405 - > Once you've given it an identity, you want to classify 406 - > these, right? 407 - > I mean, they're what, I don't know, 100 billion URLs on the 408 - > internet. 409 - > We don't manage these one URL at a time. 410 - > We're not going to manage 10 million agents one agent at a 411 - > time.

412 - > So you classify and say, this is an HR, this is a this is a 413 - > benefit, so this is whatever, travel expense, something like 414 - > that. 415 - > So that you can put appropriate policy. 416 - > So that's step one, which is literally the act of giving it 417 - > identity. 418 - > The second part is authorization.

419 - > Authorization is this is the scope on which you can access 420 - > whatever you're trying to access. 421 - > The key part that we this is the lesson that we've learned 422 - > building agents ourselves, and then how we are helping 423 - > customers together working with WWT is that authorization needs 424 - > to be scoped appropriately based on the intent. 425 - > Right? 426 - > Ideal case, and these are the solutions we're building, the 427 - > agent never holds an authorization token.

428 - > There is an entity, whether it's a sandbox, whether it's a 429 - > gateway, whatever, something else is holding that 430 - > authorization. 431 - > So IT and security can determine this is too much, this is too 432 - > little, this is appropriate, not appropriate. 433 - > But you ideal state, you never want to give the authorization 434 - > token directly to the agent because it will go hay. 435 - > SPEAKER_01: So Raj, to that point, what happens when the 436 - > agent isn't within your organization?

437 - > Let's let's say a consumer tool where I'm using my agent on my 438 - > behalf, it's calling my bank or it's ordering something. 439 - > SPEAKER_03: Yeah, so third-party agents in that authorization 440 - > becomes even more pronounced and even more important. 441 - > And the way to work this is what is called self-attenuation. 442 - > I don't want to go technical on you guys, but just very briefly.

443 - > Well, you got Brian here. 444 - > Yeah. 445 - > So maybe for Brian. 446 - > I slept at Holiday in Express last night.

447 - > Very briefly, you it let's say you have a. 448 - > So now it's using an authorized. 449 - > Topen on my behalf to go to Amex. 450 - > Yeah.

451 - > Yeah? 452 - > So every two weeks, expenses get filed. 453 - > Four, six, eight, ten, whatever. 454 - > Six months into it, there is a new credit card that is issued 455 - > in my name to send to a different address.

456 - > Is it right? 457 - > Is it wrong? 458 - > I don't know. 459 - > Right?

460 - > I might have changed addresses. 461 - > Maybe I lost my wallet. 462 - > Maybe the card expired, whatever the story may be. 463 - > But but this is what goes haywire.

464 - > I don't know whether that card is okay or not. 465 - > But when you take this read-write permission for my 466 - > scope that I've given in Conquer and take that read-write 467 - > permission to your Amex, that becomes a problem. 468 - > So you want to trim that down to just read permission when it 469 - > goes to MX. 470 - > Okay?

471 - > And when MX were to call USPS to look up if this address is 472 - > really true, you'd trim it down even further to say you only 473 - > have view capability. 474 - > Right? 475 - > So trimming down the scope is how you keep these things in 476 - > check. 477 - > Otherwise, they have absolute sort of agency to go in any 478 - > direction.

479 - > SPEAKER_02: Just another thought on the on the topic of policy 480 - > and risk. 481 - > Maybe a a different angle on it than the traditional secure, you 482 - > know, security and compliance side is there was a story this 483 - > week around an organization that unleashed their enterprise AI 484 - > tooling to their workforce without caps, without with no 485 - > metering. 486 - > And they've got a$500 million bill. 487 - > unknown: Yeah.

488 - > SPEAKER_02: Is it true? 489 - > That wasn't that was in the news this week, right? 490 - > Now it might be marketing fluff, but it I think what it 491 - > represents is FinOps starts becoming a very meaningful part 492 - > of this. 493 - > And a lot of the tools that we're deploying today, they're 494 - > SaaS.

495 - > In many cases, if you're building your own, you might be 496 - > deploying that in a NeoCloud and a hyperscaler. 497 - > It's off-prem, it's somewhere else. 498 - > But when it becomes a bill, it becomes very real. 499 - > And to your point, whether it's assigning an identity or a 500 - > policy to that identity for an agent or to the actual workforce 501 - > at large, the user base, the real people, and how they're 502 - > building and deploying new capabilities using these tools, 503 - > there has to be a mind's eye towards the not only the cost, 504 - > the you know, the token at the tokenomics associated with it, 505 - > but also there is a right size for right fit type of an 506 - > approach here as well.

507 - > And that's again back to infrastructure. 508 - > You you're likely going to see a significant shift towards a 509 - > hybrid IT, hybrid AI model that is a balance between I'm 510 - > consuming some from the public, private, as a service, et 511 - > cetera. 512 - > But I'm gonna I'm gonna have applications and workloads and 513 - > data access and security and governance controls of the right 514 - > order for the right purpose. 515 - > And I think that's, you know, maybe a few steps downstream 516 - > from where we are today.

517 - > But that that first article of a half a billion dollars in 518 - > expense wakes everybody up. 519 - > And so now there's a, you know, it's at least an acknowledgement 520 - > as of today, you have to think about this. 521 - > Yeah. 522 - > And even though you're gonna go this path now, because it's the 523 - > path either of least resistance or speed, you have to be mindful 524 - > of not only the expense, the risk, and the security, but how 525 - > you architect systems for the future to again enable the right 526 - > fit for the right, the right purpose.

527 - > SPEAKER_03: Yeah, and locally deployed models are definitely 528 - > gonna be there. 529 - > Domain-specific models, I mean, it's a little bit of an 530 - > oxymoron, but small language models, all of these are very 531 - > much part of the techniques for development at scale. 532 - > It is happening now, it's not future tasks, right? 533 - > It is because you're not gonna go to the frontier model for 534 - > every single question.

535 - > That should be your, I don't want to say last resort, but 536 - > that should be for the most sophisticated of the things that 537 - > the local models cannot adequately answer. 538 - > This is why e-vals become really important to you. 539 - > This is why we we're really happy to have something like a 540 - > Galileo as part of our mix because how do you make sure 541 - > that when an agent fails at a certain task, for you to know 542 - > what why did it fail? 543 - > Right?

544 - > So tying exactly to the spend and everything else that is 545 - > happening, there is going to be specialization of these the way 546 - > models are. 547 - > And I, to anybody who disagrees with me, I tell them just look 548 - > at how many, how much of specialization there is in 549 - > databases, right? 550 - > That's right, right? 551 - > Just look at how many bespoke variations of databases because 552 - > at scale, you cannot not have that kind of specialization.

553 - > SPEAKER_01: So absolutely spot on. 554 - > And so really honing in on what outcomes you're focused on and 555 - > what is the true cost to get to that outcome, I think is 556 - > becoming a lot more important these days. 557 - > SPEAKER_03: Accomplishing the task, yeah, which is what EBAs 558 - > measure. 559 - > SPEAKER_00: Yeah, I think 100%.

560 - > Yeah. 561 - > It's a perfect way of so not just not just token spend, but 562 - > but execution. 563 - > And to build on your news article, I'll offer you another 564 - > one. 565 - > There was a Wall Street Journal article that that found only 18% 566 - > of spending on AI coding tokens translated into shipped 567 - > products.

568 - > So, Joe, I mean, what's what do we think the gap is there? 569 - > Is it just not knowing how to use the tools? 570 - > Is it training? 571 - > Is I I think a lot of people are playing with it.

572 - > SPEAKER_01: Yeah, and they're creating stuff. 573 - > And to them, maybe it is providing value in their own 574 - > day-to-day. 575 - > But as you start looking, this goes back to sort of that 576 - > outcome-based or kind of business value. 577 - > What are you really trying to achieve?

578 - > And maybe that's more of a team or department structure. 579 - > Is it producing the outcome that group needs, not just the 580 - > individual? 581 - > SPEAKER_02: I think I think we're still really early. 582 - > Yeah.

583 - > The shift is underway, but it's you know, it's top of the first 584 - > right now for a baseball analogy. 585 - > I think there's still a long way to go for an organization to see 586 - > meaningful gains from a growth, an operational efficiency and or 587 - > innovation standpoint. 588 - > But there are absolutely whether it's 18% or 5% or 30%, I'd say 589 - > instead of worldwide, we're seeing very meaningful progress 590 - > through all of our departments, whether it's finance, it's HR, 591 - > it's ops, it's global supply chain, it's our technology 592 - > teams, all of us, our API, we're all gaining ground, becoming 593 - > more operationally efficient.

594 - > I think we're actually seeing a path to revenue gains as a 595 - > result of a lot of work that we're doing and the way that we 596 - > show up with our clients. 597 - > Now, the question is, does that efficiency translate into 598 - > meaningful progress? 599 - > Or did I just give somebody back 20 minutes and an hour and 600 - > they're gonna go screw around? 601 - > Yeah.

602 - > Right. 603 - > Is that actually translating into that 20 minutes gets 604 - > applied to a more meaningful task now that they couldn't 605 - > otherwise get to? 606 - > I think for us and for most of the clients that we encounter, 607 - > there's more work to be done than they have people and 608 - > resources. 609 - > So there's a backlog that's a million miles long.

610 - > Ideally, because we're able to get through more tasks faster, 611 - > we're able to conquer that backlog more quickly. 612 - > And again, turn your mind's eye, your best people, towards what 613 - > are the biggest problems you have to solve and how do we 614 - > start making progress against those? 615 - > SPEAKER_03: Talking about big problems and how tokens get 616 - > utilized in the eight odd weeks, eight some weeks that we've had 617 - > access to mythos, we have gone through nearly two billion lines 618 - > of code around 50 plus products.

619 - > All of that consumes tokens. 620 - > Yeah. 621 - > So it's not the producing more code, but all of the analysis 622 - > that goes into it so that you're fixing it. 623 - > Just give you one example because Mythos is on everybody's 624 - > mind.

625 - > But there are many, many such examples where there is, of 626 - > course, there is some leakage where people are filipping with 627 - > their use of tokens. 628 - > But I would say by and large, it is used for whether it's for 629 - > research, competitive Intel, digital twinning. 630 - > There are thousands of other things that software requires 631 - > other than just a writing code uh for it to go into production. 632 - > SPEAKER_00: So the measurement question matters.

633 - > How do you know AI is actually delivering value and not just 634 - > consuming tokens? 635 - > That's a real challenge, but there's a harder challenge 636 - > sitting underneath it because the same capabilities that make 637 - > AI coding assistance powerful also make them dangerous in the 638 - > wrong hands. 639 - > SPEAKER_01: I think Mythos came out and it either I think a 640 - > third of the group got completely scared and goes, oh 641 - > crap, what are we gonna do with this?

642 - > A third thinks, eh, I don't know how it's gonna affect me yet, 643 - > and a third just don't know how it's gonna affect them. 644 - > So there's definitely still a lot of confusion about it. 645 - > I know our security team acted very quickly on it. 646 - > We've we've created workshops and briefings on in the past six 647 - > weeks just because of the amount of customers asking us about it.

648 - > But we see we see it being real. 649 - > I mean, this thing, it the cat's out of the bag once again, and 650 - > this thing's got some real power behind it. 651 - > I think kind of how you guys are using it for good of hey, I can 652 - > look at all my code bases and quickly figure out what gaps I 653 - > might have and remediate them faster than maybe you ever had 654 - > before. 655 - > I think everyone's just scared of what happens when mythos 656 - > starts getting in the hands of the bad actors at some point, 657 - > and what's that gonna do to everyone's infrastructure and 658 - > their in their in their code?

659 - > But there's there's just a lot of conversations being had about 660 - > it and people need to take it seriously right now, for sure. 661 - > SPEAKER_02: And I think the just to tie it back to native 662 - > engineering, there's there is a path here where you're using it 663 - > to scan your code base or your infrastructure and look for 664 - > vulnerabilities. 665 - > To be able to patch those at the speed that you need to using 666 - > legacy approaches might just not be fast enough.

667 - > Yeah. 668 - > And so to have coding assistants at your disposal that are 669 - > proficient and actually tailor-mage for that type of a 670 - > use case to quickly remediate your code and get yourself into 671 - > a position where you're no longer at risk. 672 - > That's actually where these two worlds collide. 673 - > So now I've got tooling, whether it's mythos or other.

674 - > There are plenty of other tools out there that will help 675 - > evaluate the infrastructure and code for vulnerabilities. 676 - > And now I've got assistance to help me patch or remediate 677 - > really quickly. 678 - > I think that's again, it's changing the game. 679 - > These are not the same problems that we were dealing with 680 - > before, and the solutions aren't going to be the same either.

681 - > And we're we're coming up with new ways and new innovations to 682 - > ensure that we're staying a step ahead. 683 - > Yeah. 684 - > SPEAKER_03: I mean, you guys will see. 685 - > We've blogged about this, we've taken some of these things to 686 - > open source.

687 - > It's called Foundry Spec. 688 - > What is the harness that we built around? 689 - > Because I mean, 1.8, whatever that number is, 1.

8 billion 690 - > lines of code. 691 - > You don't go to a chatbot and say, here is a GitHub repo, tell 692 - > me the button. 693 - > That's not how it works, right? 694 - > So there is a lot that goes into the harness to make sure that it 695 - > is accurate.

696 - > Cisco's taken that, what we used internally, and open sourced it, 697 - > right? 698 - > So everybody can use it. 699 - > And I know I've talked to to Brian and a couple of other 700 - > folks within your organization of how best to use it. 701 - > The thing that I wanted to say again there is yes, finding 702 - > bugs, yes, fixing code, but the biggest benefit that we get out 703 - > of the AI model is the richness of the test cases that are 704 - > built.

705 - > Because it's those test cases that help us make sure that this 706 - > fix is a durable, nothing more, nothing less. 707 - > The performance is accurate, and that we have genuinely plugged 708 - > it. 709 - > It's not like we've just taken care of one aspect of it, but 710 - > this is going to be effective in many different ways that we are 711 - > going to see the attack form. 712 - > So, from my perspective, not even just a practitioner's 713 - > perspective, my perspective, it is the ability to write a very, 714 - > very rich library of use cases where AI is a huge asset, more 715 - > than even fixing the code.

716 - > SPEAKER_00: So we'll close out with this question. 717 - > Um, I'm gonna ask each of you, you know, just in general, what 718 - > do you think a priority or a you know a tip or piece of advice 719 - > is? 720 - > Joe, I'm gonna ask you from the perspective of a user. 721 - > Brian, you'll be from the perspective of an organization 722 - > or an executive.

723 - > And then Raj will ask you from the security standpoint, what do 724 - > we need to be doing to make sure that the back half of 2026 we're 725 - > set up for success with coding agents and AI native 726 - > engineering? 727 - > SPEAKER_01: Yeah, I'd say from the user perspective, it's be 728 - > curious, start testing out these tools, play with them, become 729 - > familiar with go get training. 730 - > If your organization isn't providing these yet, you know, 731 - > start fighting for those licenses because this is the new 732 - > way.

733 - > And if you're not doing it, your competition's coming up on you 734 - > because they're using the tools too. 735 - > So yeah, get used to it, don't be scared and just dive into it. 736 - > SPEAKER_00: Okay, so go go rack up the tokens. 737 - > So go rack up the tokens.

738 - > Okay, Brian might have something to combat that with here. 739 - > Yeah, maybe. 740 - > SPEAKER_02: No, I think you know, if I'm an organizational 741 - > leader or an enterprise and a business leader, I think one of 742 - > the things that I'm as I just reflect on our conversation and 743 - > all the conversations that we have, events like this and 744 - > others, it's a lot. 745 - > There's an awful lot to deal with.

746 - > Whether you're in purchasing, you're a finance leader, you're 747 - > a CIO, CTO, CISO, chief data officer, CEO, board. 748 - > The the amount of demands and things to unpack all at once, 749 - > it's never been at this kind of pace. 750 - > And the margin for error is just too narrow, right? 751 - > It's just it's a combination of factors that make it very 752 - > difficult for an enterprise leader to be able to move at the 753 - > pace that they'd like to keep up with or even advance themselves.

754 - > I I think the most important thing is, you know, being in an 755 - > event like this with Raj with Cisco, choosing the right 756 - > partners matters. 757 - > And it makes an just an immediate impact when you look 758 - > at having the right organization at your side to help you 759 - > navigate the complexity of all these things, learn from each 760 - > other, benefit from the experiences that have been 761 - > getting elsewhere, tackle new problems in new ways. 762 - > But I think, you know, organizations like Cisco, like 763 - > worldwide, I think we we we pack the goods to really help 764 - > navigate the complexity of all the challenges that exist out 765 - > there and do so in a way that we're gonna fail quick, but keep 766 - > moving forward and and ultimately find a really 767 - > positive outcome in the end.

768 - > SPEAKER_00: Yeah, so Raj, I mean, if if we want everybody to 769 - > get our hands on this, but also the margin for error is thin and 770 - > there's some risk here, what do we need to know from a security 771 - > perspective? 772 - > SPEAKER_03: I would say three or four things. 773 - > Number one, you guys run capture the flag exercise. 774 - > Yes, take advantage of it.

775 - > Yeah, right. 776 - > There's never been a better time using the sort of open source 777 - > sort of tooling that is out there. 778 - > Capture the flag, really important. 779 - > Run a hackathon.

780 - > Run a hackathon like you actually mean it. 781 - > Give people truly two days, whatever the time frame is, and 782 - > and give them the ambition to be bold. 783 - > What is the biggest problem you would solve if cost was not an 784 - > issue? 785 - > Right?

786 - > The third thing I would say is you've got to celebrate that 787 - > moment where something happens. 788 - > In if you use general terms, things become like, yeah, okay, 789 - > we'll get to it one day. 790 - > Here was how I inspired our team. 791 - > We were at a customer advisory board, we ran a session, I told 792 - > everybody explicitly from 3 30 to 5 30, we were gonna say, What 793 - > is the most important new feature you require?

794 - > Right? 795 - > And the goal was that by the time CAB was over, which was a 796 - > day and a half later, we will have that production ready 797 - > feature. 798 - > That was an explicit goal. 799 - > You're gonna get something that you don't know about between 800 - > 330, between 3:30 and 5.

30. 801 - > And by the time they leave the day after next at 11 or 12, it 802 - > is has to be in production. 803 - > And we did it. 804 - > By making it concrete, people know what is the what is it that 805 - > that makes it real.

806 - > Otherwise, like code, we could go faster, blah blah blah. 807 - > Like those things are good, but they don't have the impact. 808 - > Yeah, give a goal that is concrete and then celebrate it. 809 - > Really important.

810 - > And then if something fails, yeah. 811 - > Whatever. 812 - > Fix fast. 813 - > Uh failure will happen.

814 - > Move on. 815 - > Um but this is the moment time to be precise and auto. 816 - > SPEAKER_00: All right, thanks to Raz, Joe, and Brian for joining 817 - > us during a very busy week at Cisco Live. 818 - > The key lesson here: AI native engineering begins with 819 - > experimentation, but it can't end there.

820 - > The organizations that gain the most from these tools will not 821 - > simply generate more code or consume more tokens, they'll be 822 - > the ones turning domain expertise into working 823 - > solutions, measuring whether those solutions produce 824 - > meaningful outcomes, and building the security controls 825 - > required for systems that can reason and act independently. 826 - > That means giving people room to experiment. 827 - > It also means giving agents identities, limiting their 828 - > authority, understanding their intent, and knowing when their 829 - > behavior begins to drift.

830 - > And perhaps more importantly, it means choosing problems worthy 831 - > of this moment. 832 - > This episode of the AI Proving Ground Podcast was co-produced 833 - > by Nas Baker, Kara Kuhn, and Sarah Chiadini. 834 - > Our audio and video producers were John Knock and Brian 835 - > Gagliano. 836 - > My name is Brian Phelps.

837 - > Thanks for listening. 838 - > See you next time.