PwC's Chief People Officer on Training 80,000 People for the AI Era With Human Skills at the Center

Speaker A: Feel your best and amplify your everyday look with Thrive cosmetics. Go to thrivecosmetics.com shine uh, 26 for an exclusive offer of 20% off your first order. That's Thrive Cosmetics. C-A U S E M E T-I C S.com shine 26 hello everyone.

Speaker B: Welcome to another episode of Future Ready Leadership. My guest today is Yolanda Seals Caulfield. She is the US Chief People and Inclusion Officer over at PwC. Yolanda, thank you for joining me.

Speaker C: Happy to join you Jacob. Thanks for having me.

Speaker B: Oh yeah, I'm looking forward to the conversation. We're going to be talking a lot about, uh, AI, adoption, people, culture, all sorts of fun stuff. Can you first give us just a little bit of background information about PwC in the US how many employees do you guys have? And maybe a little bit about yourself and what you do there as well?

Speaker C: Sure, I'm happy to. So PwC US is uh, one of our member firms. We are a network of firms across the world From for the US firm we are 80,000 strong. About 40 plus thousand of our people sit onshore, meaning in the US and then another 40,000 sit offshore, meaning in what we call our acceleration centers. So they are our centers that sit in India, in Argentina and other countries around the world. So we are an incredible population of talented professionals, many of whom join our firm directly out, uh, of undergrad and certainly and then stay throughout the course of their careers through partnership for some and others will leave and enter industry and do great things. I joined the firm 16 years ago almost as a direct admit partner. I'm a lawyer by trade, so I spent the first 25 years of my career as an employment lawyer. And I joined PwC in 2010 as the Chief Employment Counsel. Throughout the course of my time with the firm, I spent so much of my time with the Human Capital team and with the inclusion team and what we call our partner affairs team, which is human capital for partners. And about six years ago I moved over to join the Human Capital team and it has been an incredible experience and the opportunity to really wake up every day thinking about the experience that we are creating for our current and future PwCRS.

Speaker B: Very cool. Uh, I also had your chief, uh, AI officer Dan Priest was a guest on the podcast a little while ago. Um, who else did I have? I think I had a few folks over the years. Tim Ryan was a guest on the podcast a while ago as well. So we go way back. I had a lot of the PwC crew, uh, on the show over the years. So it's interesting as we're recording this, I don't know if you saw the announcement today, but Anthropic just released their brand new model Fable. And so it's becoming harder and harder to keep up because, uh, Opus 4.8, I believe, came out at the end of May, and now here we are literally two or three weeks later, and there is this new crazy model that is out as well, making the news. It's trending on X. Everyone's talking about it, it's going everywhere. Uh, so talk to me a little bit about what you're doing with AI and let's start very high level first on maybe how you're approaching it, if you're able to share which models or tools you're using. And then we can dive a little bit deeper into some of the specific topics.

Speaker C: Sure, Jacob, I'm happy to. So one of the things that we did really early on as AI came on the scene in such a significant way was we made a couple of decisions very early on. We said we need to invest in upskilling our people and we don't need everyone to be an AI expert right away. But we needed all of our people to have a level of comfort with the technology so that they understood how to use it, they understood how it might apply to their work. If they were going to be deep domain experts, they really understood how to build with it. And if they were going to help our clients with this transformation, they need to understand how to sell it. And so different people needed different levels of understanding around AI, and we did that early. The second decision we made is in order to drive adoption, in order to truly understand the value that this technology can have to our organization and our clients, we need to democratize access to the tools. So every PwCr has access to chat. PwC, which was our first, our first and I think the first, um, kind of internal enterprise rollout of ChatGPT. We then, um, gave every one of our people access to copilot. So every PwCr has ChatGPT, has PwC, rather has copilot. Um, and then we also have access to all of the other tools that are available and you have access to the tools as you need them. So you talk about, um, the new rollout with Claude. I haven't used it today, so I haven't had a chance to sort of experience it. But we have not only access to these tools, but every new rollout is sort of pushed out to us automatically. And so by putting the tools in the hands of our people, we have enabled them to use it as we continue to change our workflows and change our methodologies and change the way we deliver to our clients. But we've also created the space and the freedom for the experimentation and the curiosity that we want everyone to exhibit when exploring these new tools. So we went out very quickly to drive access to AI and to build a baseline level of training. And I'll tell you, 90 plus percent of our people engaged in the early training. They really sort of invested in themselves and made a commitment for their own growth. And so that was our broad approach to AI. It now gets more specific as our needs change.

Speaker B: It's funny, I remember, um, didn't you, a few years ago, I think it was with you, you had a digital accelerator or digital innovation program. I actually keynoted, uh, several of those events that you did. What year was that? Do you remember how long ago that was? Oh, my gosh, it was a while, right?

Speaker C: It was pre pandemic, so it was like maybe 16, 17, 18.

Speaker B: Yeah, yeah, yeah. So it was interesting because I remember even then, um, when I was speaking at the events, and I think we did three or four events together. I worked with Sarah McEnany.

Speaker D: Yeah.

Speaker B: Um, and I can't. There were a couple other folks on

Speaker C: your team and Jo Atkinson and those folks.

Speaker B: Uh, yeah, yeah, yeah, yeah. And so we did a series of events. And I remember even then, obviously the topic of AI was coming up. It wasn't nearly at the scope that it was now, but one of the things that I distinctly remember from those series of events was that even almost a decade ago, you had this program in place before AI became so mainstream, where you were encouraging your employees to, to be. Challenge the status quo. And from what I recall, the program was literally, you had employees on different teams who, when there was a specific process being done a certain way, or a client was being serviced in a certain way, you had employees who would literally say, hey, wait a minute, maybe there's a better way we can do that. Let's try to do it this way. AI was a part of that conversation, of course, but still, even a decade ago, you were already challenging how work got done.

Speaker C: Absolutely. And when I think about what we did with our digital accelerators, and you're talking about our digital labs that we had set up everywhere, it really did create the very clear blueprint for how we've driven AI adaption adoption, um, and usage today. So, um, when I think about our digital acceleration journey, we went out really boldly and we said, we are going to go on this upskilling journey together and no person will be left behind. That is the exact same thing that we said 10 years later, um, about AI. We are going to go on this upskilling journey together and no person is going to be left behind similar to what we did back then. We've also thought a lot about how do you gamify this, how do you amplify and tell the stories in a different way? How do you reward people for that grassroots ingenuity that we need? That's a bit of what we are designing today with different awards and different pop up labs to help our people become more AI native as we continue to grow in this space.

Speaker B: But one of the things I really appreciated is that you were doing this before AI became the scope that it is now. So to me that just shows that for you this wasn't an AI strategy. This was just a general culture and people strategy before AI even became mainstream. Whereas I think a lot of companies now, they waited for AI, they're freaking out and now they're trying to introduce a similar program to what you have. But you did it a decade ago before. In other words, you didn't wait for uh, the catalyst. You kind of saw the future a little bit. Um, and I suppose that makes the adoption and the implementation and the conversations around this much easier because you've already been talking about it for a decade.

Speaker C: I think that's right. I mean one of the things that's unique for us and not unique to PwC, but probably unique in professional services environments is that we hire thousands of students off college campuses every year. And so when we say our ambition is to be the world's premier developer of talent, we mean that in every aspect of the word. And we have this very unique role that we play in training entry level talent. And when you're training entry level talent, you have the responsibility to help people build incredible skills and careers that will either take place at PwC or will take place when they leave. And we certainly Enjoy a tremendous PwC alumni network that are, you know, leaders in industries all over the world and all different, um, in all different industries you will find PwC alums because we do train people so well here. The beauty of that for us, when you think about new technologies like AI or other digital accelerators or other different platforms, we become kind of client zero for ourselves. We have a tremendous amount of support to invest early and quickly in introducing new technologies to our people and driving new ways of learning and Driving upskilling, because we not only need to figure it out for ourselves, but we want to position our people to be able to help figure it out for our clients. So it puts us in this really unique position where we have, um, not only the latitude, but, in fact, I would argue the mandate to go out early and very quickly, understand how these technologies can be used, how they can be used responsibly in our environments, how to upskill our people, how to train them differently. But learning and development is just so core to who we are and what we do. And I think that's why you saw us doing this 10 years ago, and that's why you'll see us doing it 10 years from now with whatever is becoming ubiquitous at the time.

Speaker B: So your two main models, it sounds like, are, um, coworkers and chatgpt, Is that right? The kind of the two that you're giving your employees.

Speaker C: So copilot. Our people are copilot, and they have chat, PwC and, uh, chatgpt. Um, but they also have access to pretty much every tool that is out there. There's some segment of the PwC population that has access to it. One of the things that we introduced recently, Jacob, was a dashboard that now will tell you which tools you have access to, what other tools are available, and how to request access, and it will also tell you how frequently you are using AI. We created our own streaks where you can go in and say, I'm at day 50 of consistent AI usage, because we want our people to understand how they're doing, but also how they compare to their peer set. I can also click on a dropdown button and it'll tell me who in my ecosystem are the top 10 users. It won't surprise you that my CEO's name is up there in the top 10 users in terms of being really driven to use the technology every day. But helping our people understand what tools they have access to, what tools are still available for them, and how to get access to those tools is something that's been really critical for us.

Speaker B: Okay. And I'm gonna. I just took a note here. I'm gonna ask you about that in just a minute. Um, so the models that your employees are getting access to, are they the frontier models? They're kind of the latest models? Or are you maybe one or two generations behind, as you kind of test? Or is it just whatever is new? That's what they get.

Speaker C: Whatever is new is automatically pushed in our ecosystem.

Speaker B: Okay. Um, how do you deal with. And have you had to deal with any mistakes, hallucinations, um, you know, issues or challenges with the new frontier models. I don't know if you have any examples or stories that come to mind, but I find that there are sort of two camps of organizations. There are some like PwC who are kind of, we want the latest and greatest tools. If there's a mistake or a problem, we're going to figure it out as we go through it. And then there are other companies who are more like, well, we're going to be one or two models behind. We're going to have really strict guidelines and regulations and governance. We're going to go slower, but we're going to be safer. But it seems like you can't have both. You can't be flying at the speed of the frontier models and also have the most secure and the best governance at the same time.

Speaker C: So I would argue that we have managed to do both and here is why. Jacob so we are giving our people access to the latest in the frontier models and as quickly as our um, chief technology officers can get through the new models, test them to make sure that they're going to work in our environment, they'll be safe for our data and safe for our client data. They push them out. But one of the things that we've done at PwC is we've told our people very, very clearly that we want you to have native and incredible AI and technical skills. We also want you to have incredible human skills. And it's the human skills that brings the judgment and the critical thinking in the context to anything, to any artifact that you are getting from AI. So we do not expect our people to put their problems in the latest clawed technology or the latest um, ChatGPT or the latest copilot technology and have it produce an artifact that they then hand off to their clients or hand off to their leaders in any way. We expect our people to bring their human judgment, to bring their human ingenuity, to bring their human context and understand the complexity of problems to bear to make sure that they are producing something that is worthy of the PWC name. And so by teaching AI and human skills together in every instance, we are reinforcing that message at every step of the way with our people. I think that's why we haven't had issues. Um, I think we are very careful, we are very protective of our data and of our clients data. We want to make sure we're using these things safely in our environment. But we teach people the responsible use of AI first. That is the first training course you're going to take here, um, you're going to get that training in your onboarding, even if you're coming off college campuses, and we know that you're more AI native, the very first thing you're going to do in onboarding when it comes to, to AI is understand the responsible use and then understand how really, really critical the human skills are that go along with it.

Speaker B: Okay, and you kind of answered my question there as well, because you said that you're. So your team does testing before it gets rolled out. So let's say Claude, uh, Fable came out today. It's not as if the second that employees turn on their computer, they have access to it. It gets tested through your team first. Okay, uh, how long?

Speaker C: If I open Claude right now, Fable is not in there.

Speaker B: Okay. And so how long is usually the testing program? Can you share any insights on that? So you have several CTOs that kind of get together, they go through it. It's like a couple days a week.

Speaker C: I couldn't tell you exactly how long the programming works. I can tell you that we have this really broad base of CTIOs operating across the firm. So we have a central team. And that team, as you would imagine, is responsible for, for how things come into the organization, how they're tested, both in the US and they also sit globally. And then each of our lines of services have a CTIO office, and their jobs are to determine how to use this technology in the delivery of the services that they provide to their clients. And then we have a governance mechanism, as you would imagine. And so all of these leaders come together in a one firm governance perspective to make sure that we are advancing the right levels of frontier relationships, that we are using the technologies in the right way, that we are getting access to the technologies in the places that need access, um, to the technologies and that we are working with. And they're then working with human capital around how we drive development and adoption of these tools. So there's a broad group of individuals that are working on this, um, pretty much every day.

Speaker B: Okay. Uh, and then once a new model is rolled out, um, so, for example, I recently gave a talk at MasterCard, and one of the things that they said, their chro and their chief Learning officer, is that a lot of the models and the plugins or updates, they're coming out so quickly that they don't always have time to create training modules and courses to roll out to employees, because by the time they do, there's something new. So look at Opus 4.8 that came out at the end of May. You know, it takes what, uh, a couple days or a week to create the training program, to roll it out by the time people. And all of a sudden there's a new model. And so if you're using several models like ChatGPT and Claude and Cowork, you're dealing with a new model or a plugin or a feature every week. So how do you manage and deal with that? Because it's not possible to create a module or a training program every time you have a new model. But at the same time, employees need to be up to speed on them. So how do you deal with that challenge?

Speaker C: It's a great question. I do think when you think about learning and development, that traditional way of there's going to be a new technology, you're going to build a boot camp, you're going to then roll it out. We are moving way too fast for that, for all the reasons you've just said, Jacob.

Speaker E: Right.

Speaker C: So that just doesn't work anymore. So how we teach our people has to change. So we need to build learning that happens in the flow of work. You need to build learning that is happening in the technologies. I mean, one of the most incredible things about AI is I could go in the cloud today and say, hi,

Speaker E: Ryan Reynolds here for Mint Mobile. Are you looking for a beach read this summer? May I suggest your big wireless bill? It's got some suspense, mystery, a slightly flat emotional arc and a shocking twist where you realize you've been overpaying the entire time. Fortunately though, Mint's story is better. Every plan, $15 a month, even unlimited. That's it. Happy ending, zero tears. Give it a try@mintmobile.com Switch upfront payment

Speaker A: M of $45 for three months, $90 for six months or $180 for a 12 month plan. Required $15 per month equivalent taxes and fees, extra initial plan term only greater than 50 gigabytes. Me slow when network is busy.

Speaker C: See terms tell me what is new with Fable and how to use it in the way that I've been using it. And because it knows the way in which I it will give me as good of a training and understanding around that as anything that my people could build in a boot camp that would be rolled out two weeks from now. So the technology enables a much faster way of learning and upskilling, which I think is incredible. But then we're also thinking about learning really differently. How do we embed learning into the workflow so that our people learn while they're doing. And that's some of the work that's being driven with L and D and our CTIOs now. But then the third thing that we've introduced recently, um, ah, are these sort of pop ups that we've been doing across the country. Because at the end of the day we still need to meet people where they are and give them the tools that they need to learn how to use this technology in the right way. And so we've introduced these AI coaching live pop ups and they move around from office to office and individuals can sign up, they can sign up individually, they can sign up with their teams. We like them to tell us what they're working on so that we have the right, right AI coaches and experts in the room. But it enables our people to come to this pop up. Think of it as like a genius bar, um, and get help. And so whether you are coming to us as an employee who has a really deep expertise in AI and you're trying to solve a particular particularly thorny problem for, for the firm or for your clients and you're trying to understand how to use the latest um, release to help you do that, we'll have people in the room that'll help you do that. Whether you're someone who's saying, look, I have access to all these tools and I experimented with them, I'm not quite using them in my day job yet, I need to think about how to do that. We'll be there to give some coaching and help them do that as well. So these have been going really well. They pop up city to city. We send people notice that we're coming to town. We have been over subscribed at them, um, at every instance and we are there to meet people where they are. So that I think is a really important tool because at the end of the day this technology is moving fast and furious. We are becoming more AI native every day. So there's some segment of our population that's going to dive right in and not have any trouble adapting to the new tools. But we also want to make sure that we are giving people support and a more white glove service where they need it.

Speaker B: You mentioned something about embedding the learning into the flow of work, something that you're currently working on with uh, your team. Can you unpack that a little bit and how that works, what that might look like?

Speaker C: Yeah, so we're piloting a couple of different technologies and we're piloting ones that are being built in the external environment. We're piloting building them ourselves. But think about it this way. Um, as our business people and our technology leaders are thinking about how we deliver the work that we do in a different way, and they're thinking about how we embed AI into the work that we're delivering, one of the things that we'll have to do is say, take this new methodology. So if we are building a new methodology around how we do valuations and deals in an old world, we would take that new methodology, we would go away, we would build a boot camp, we would deliver it to our people, we'd get them trained up and they'd be ready to go. We need to move much faster than that. And we're enabled to move much faster than that because the technology is in place that enables that. So what we are building is how do you take the methodology and actually build learning into the methodology? Have people actually doing the work using the new methodology with pop ups that might say, have you thought about this? Have you evaluated this against this? Here's an opportunity to apply this critical judgment skills. Oh, by the way, let's actually pause for a moment and test your proficiency in using AI in this space or in this particular skill. So the notion is that you won't have methodology that sits over here and learning and development that sits over here. They have to coexist in this space. That's what we're, that's the, that's learning in the future. That's what our teams are building.

Speaker B: So while you're. So let's say I'm doing something on my laptop, I'm working on a presentation or whatever. So it's almost like there is some sort of AI that sees and observes the work that I'm doing. And based on that work, I get a pop up that says, hey, you know, we noticed you're working on this presentation. Did you know that there's this cloud plugin that you can download into the presentation, just upload the branding guidelines and it'll take care of this all for you?

Speaker C: Absolutely, absolutely. And it might help you and say, do you now want to practice this presentation with your AI coach and let us pull that plug in here and help you to practice it so that you have an opportunity to grow those really important storytelling presentation human skills right alongside the technology that you're building. That really is the future of learning. It is real time, it is interactive, it is AI enabled. But there are still some really critical human skills pieces that go along with it and technical skills pieces, depending on where you sit in our business. But that is the future of learning. Now we'll still have enterprise wide learning where we're bringing people together in classrooms because that kind of community is really important. But we also want to understand that most of our learning happens day to day. In doing the work that we do, the apprenticeship model will be alive and well. We need to make sure that people are sitting side by side and learning from one another and teaching and learning every day. And the technology is an incredible enabler for that.

Speaker B: Is this something that you're having to build or is there a vendor that does this? Because I haven't seen any and maybe there are some startups out there, but I haven't seen any, uh, technologies specifically geared towards that. So you kind of building this yourself internally?

Speaker C: We are building it internally. We're piloting some other tools that are growing in the marketplace. There are new things entering the markets all the time and so it's hard to keep track. We have a few pilots, pilots, uh, that are underway. You're right, it is hard to keep track. So we have a few pilots that are underway and we're also piloting, building things ourselves as well.

Speaker B: Very cool. Um, so one of the things that you mentioned, um, is this kind of dashboard and there's this, been this trend, I'm sure you've seen it, called token maxing. And I think a couple organizations like Uber and Disney, uh, and several others have gotten in, I don't want to say trouble but because they've been publicizing the. So the way that it works is employees, they have this internal leaderboard and the employees who use the most AI, the most number of tokens, they get, get, you know, placements on these leaderboards.

Speaker C: Not sure whose wife.

Speaker B: And what ended up happening in some of these organizations is that the, the metric became the goal. And so in other words, people were just using AI to check the weather, they were using AI to look at sports scores, they were using AI to do all these very basic things that maybe they didn't need to use AI for because they wanted to get as many tokens as possible to get up on the leaderboard. So how do you make sure that the goal doesn't become just massive token usage, but the goal actually becomes business impact and business outcomes?

Speaker C: Yeah, no, um, so it's a really important question. It's interesting. It's one that we were spending some time thinking about today actually as we're thinking about how do you evaluate AI adoption in a way that isn't just purely based on how frequently people are in the technology. Because you're right, you could be doing anything in the technology. And that doesn't mean that you're getting the breadth and the depth of understanding that you need. So we are thinking about it in a couple of ways, Jacob. We are thinking about it as a performance metric. So how are we documenting impact with AI in our normal performance processes? We have a, uh, tool that's called Snapshots, which is how you get feedback from someone on an engagement or work that you've been doing. And we're thinking about how do you embed AI impact into that tool so that we evaluate whether or not this individual is using AI and what impact they're creating by having it? Because we really want to evaluate impact, we're thinking about how to embed, um, qualifiers around proficiency. So what are the conversations that people are having with their development leaders on a fairly consistent basis as they're seeking feedback where they can evaluate not only their tool usage and their impact, but their proficiency in terms of their ability to really lead and drive differences with these tools? Those are some of the ways in which we're thinking about it. So it's not just about, you know, how many times are you clicking in the tool and evaluating what you're doing with it. In my mind though, these are very temporary measures. We have a period of maybe 12 to 18 months where we need to understand and feel confident that our people are adopting and using the tools and developing the best the breath. If you fast forward 12 to 18 months from now and you think about the work we're doing around transforming methodologies and the work that we're doing around transforming workflows, AI is just going to become the way we work and it's going to be so embedded in the way we work that I actually don't have to wake up every day wondering if our people are using the technology because they won't be able to do their work without it. So there are some near term things that we have to do to evaluate usage and depth of usage and we're, we're planning and thinking about how to do that. But again, I think these are very short term fixes because soon the work will simply require that you use the technologies in order to deliver and our people will become more AI native every day.

Speaker B: That brings up kind of the uh, the ROI question, and I agree with you. I think eventually it's going to get to a point where it's like, what's the ROI of your laptop? What's the ROI of, of slack or of any of these tools, it's just going to be a part of how you work. But right now there's this really big debate around ROI of these tools. And at least the general consensus is that for individual productivity, like if you were to ask your employees, they would say, oh yes, I see it, it's great for this, it's great for that. But to measure it across the enterprise and to show here's how much money it's making or saving, companies are not yet able to do that. And there was this recent post, uh, or this recent announcement that came out from Uber where uh, and I don't know if you saw this one, their CTO basically said that they blew through their entire AI budget for the year in four months and that led them to basically throttle employee usage of AI tools. And I think for most roles that are not encoding, I think it was like 1500 bucks a month is what you're allowed to use in terms of token usage. And so other, other uh, companies are coming out there and they're saying, wait a minute, like if we're giving everybody unfettered access to these tools and they're just running uh, like our entire code base through there, they're running everything through these AI tools, the costs are ballooning so much and on top of that we still pay for the people that it's not sustainable. So how are you handling with the, the ROI conversations and the spending conversation and are you doing anything to throttle AI, uh, usage for your employees?

Speaker C: Yeah, no. It's a great question. So our CTIO teams and our CFOs offices are working really closely on this to understand one, the actual cost of AI. And I think organizations are still trying to figure that out. It is so highly subsidized today. Right. That we don't know the full cost. And look, if you read different studies, you'll read studies that would suggest that many of your entry level or offshore resources will become more cost effective than AI, um, in the future because the cost of the technology may become such that, that it is not sustainable for some of the repeatable work that we're having it do. So I think we still have some time ahead of us to figure that out. What we have tried to do is control, uh, give our people access to some basic tools every day. And I say basic, they're not basic, but give them access to the tools that we've democratized. So copilot chat, PwC, every PwC has that. The next layer of tools you need to justify why you need them. And that is our way right now of trying to manage the cost and manage the ROI and make sure that if you are giving someone access to claw, that there's a real reason that that technology is required, that what they need the technology for can't be achieved in copilot, can't be achieved in chat PwC. And so our CTIO office and our CTIO is in the line, along with their finance leaders in the line are making really discerning and, um, um, concrete decisions every day about how to strike the right balance in terms of ensuring that we're giving our people access. And it is a balance, right? Because from a people experience perspective, we want to inspire innovation and creativity and grassroots exploration. And we know that some really incredible outcomes come from that. In fact, we've created an award program just to understand the types of incredible outcomes that come from that grassroots innovation. But we have to balance that against cost and we have to be responsible in that way. So those teams are working really closely to come up with the right models to determine the impact. But our kind of gate for today, if you will, is, um, understanding the justification for tools beyond the ones that are available to everyone.

Speaker B: Okay, so then it sounds like for most employees you'll get access to something like Cowork or chatgpt. If you want something more advanced like Fable, there's a process to submit a request and then there's a conversation or something that comes up where somebody will say, hey, Jacob, we noticed you put a request for the most expensive, top of the line, uh, you know, the Hermes model. Uh, why do you need access to this? What are you doing here that you can't do with something else? And is this an individual conversation? Because you mentioned 40,000, 80,000 employees. That's going to be a lot of conversations. So are you streamlined?

Speaker C: It's more team based? Yes, it's absolutely more team based. And so there's really one individual asking for access to technology. It tends to follow the work that they're doing, and so the teams will ask for access. There certainly are instances where there are individuals asking, and we work through that, but it's much more driven by team. So if you need the technology to deliver for your client in a different way or to build a solution for your client in a different way, and by the way, where it makes sense, we very freely give access to these tools. We are not, um, overly rigid in terms of providing access because again, we want to encourage innovation, we want to democratize access to the tools. We want to encourage people to get the best outcomes for our clients leveraging these tools. But we do have some gates to make sure that we're being responsible with them.

Speaker B: Because, I mean, I have a small team that I run, for example, and I would imagine if somebody on my team started using Fable, which is double the cost, my question would be like, hey, wait a minute, like, we're a small business. I know you're using Fable. Are you sure you can't do this on ChatGPT, where you're not being throttled with any token usage? Because, uh, my AI bill now tripled for the month. And that would be a conversation for sure that we're having. Um, okay, so then I like that approach of just everyone will have access to something like ChatGPT. If you want something more advanced, it's usually on a team level. And then you start to have, uh, conversations, uh, and back and forth on that. But even at that level, are you finding. And it's probably confidential, you don't need to share. But in terms of, like, your annual AI spend, is it at a point where it's going to start to eat into any kind of margins? Like, is it going to start to be a line item on the balance sheet that's getting bigger and bigger and bigger? Because 40,000, 50,000, 80,000 employees, some of them have the Frontier models. Uh, you know, it's for your engineering team, if they're using it, 1,000, $2,000 a month, 5,000amonth per individual. Like, you can see that it starts to become all of a sudden millions of dollars every year, 5 million, 10 million that's being spent on these AI tools. So are you. Are you tracking the dollars spent on AI, um, very carefully right now, or are you just kind of, you know, letting it go and just seeing what happens with it?

Speaker C: No, I think, uh, as, uh, you would expect, like any responsible business, we have a budget for how much we plan to spend and investing in AI, and we are absolutely tracking to that budget. So there are models that our finance teams and our CTIOs have built to help us understand the cost of AI, to understand the ROI of that cost. There are clear budgets. Like, we all have budgets across the firm for our levels of investment. There are investment dollars that are dedicated to this, and they will be very closely, as you would imagine, monitored not just by our leaders, but by our board. By our board as well.

Speaker B: Okay. Okay. So you're tracking. Because I find that, at least in the case of Uber or some of these other companies, like we had a Chro Group, uh, meeting And I won't mention the company name. It's a, uh, very massive employer. And the, uh, CHR there was telling us a story about how one of their employees, they were using something like ChatGPT, and they built their own kind of like, cutesy, more friendly version to let employees use it. And they noticed that all of a sudden this fun version that this employee built was being propagated across the organization that has hundreds and hundreds of thousands of employees. And all of a sudden they started to get. They started to notice. They're like, wait a minute, this is a massive, massive spend. And they had to actually throttle it and to, uh, remove access for some people and to really tone it down because the spend got so high. So I find that some companies out there, they're so focused on the usage and the adoption that they're not thinking about it in terms of budget. And then one day they get a bill and they're like, what the hell is this? Like, where did. And then they start thinking about budget.

Speaker C: Yeah, no, no, we definitely do not have the luxury of that. Um, our finance teams, uh, our leaders who manage our investment spend, watch very closely the cost of this technology, um, because it is an investment, right? And it is sort of understanding our capacity to invest in this way. And creating the capacity to invest is something that our firm is very focused on. So it is closely monitored. It is budgeted, it is, it is an investment in our future, but it is not one that is done without the right level of guardrails around it.

Speaker B: Okay, so, uh, if somebody were to ask you, what's the ROI of doing it? Um, I know there's probably not a dollar in cents number behind it, but if somebody said, hey, Yolanda, you guys are probably spending, I don't know, millions of dollars a year on AI, what are you getting as a result? You know, why not save that money? Are you seeing something that is telling you that this is a good investment for you? You.

Speaker C: I don't think it's. I don't. I don't think we have a choice, Jacob, in this level of investment, we are in a client service business. Our clients are going to expect us to help them solve some of their most important problems, and they're going to expect us to use every tool available to us to do that. AI is clearly an incredible enabling tool that will, um, not only drive efficiencies, and I think everyone is starting to understand some of the efficiencies, but will also help drive innovation and creativity and helping us to solve our clients. Problems. So we don't see this as an optional investment. This is absolutely an investment that we have to make in the future of our firm. And those of us who are in the firm today are mere fiduciaries of this incredible organization. And we need to make sure that it is set up to be sustainable in the long term. And so we don't see this investment as optional. We do need to do it responsibly. We do need to make sure that we're providing our people with the skills that they need to be successful. We need to leverage the technology in a responsible and thoughtful way. But I don't think anyone would think that this is an optional investment for us.

Speaker B: Yeah, that's an interesting way to think about it. It's kind of like having, uh, an ERP system or having, uh, uh, an HR system at a certain point, or having a mobile phone, smartphone or a computer. There used to be conversations around why make those investments as well. And now you can't imagine somebody not having these things.

Speaker C: Right.

Speaker B: So it sounds like for you, this isn't a conversation of, well, should we make the investment or should we not? It is, we have to do it because our employees, our customers have these tools. They're going to be using these tools. And so if we're, if we're not using it, we're not going to be able to compete against, uh, you know, the other big firms out there, and we're not going to be able to serve customers in the way that they need to be serviced because they themselves are using these tools. So it's, it's probably like that, I would imagine, for every business. Like, it can't. It's probably not a choice for anybody out there.

Speaker C: Can't imagine that it is. I mean, if you think about the efficiency gains alone, just responsible business practices at some point are going to require most organizations to at least evaluate the impact of AI on their business. I think AI is a disruptor, and it will disrupt the way that our business gets done. It'll disrupt the way many businesses get done. And so thinking about that and planning for it is in fact an obligation of most, uh, leaders, regardless of industry.

Speaker B: And I think the consulting model especially is very much ripe for disruption in this area because you're not going to be paying, uh, a McKinsey or whatever, $10 million to do this crazy big project now when you can go into fable, maybe spend 500 or $1,000 and get as much of a detail analysis. So it's disrupting a lot of different Industries and areas, professional services, finance, wealth, advisors. Uh, I mean, I'm speaking at a conference next week for dentists. Uh, and even there the conversation of AI is coming up. So it's disrupting, I think, every model and every business you can think of.

Speaker C: I think that's right. And what businesses are doing, and I know we certainly are, is thinking about how do we, how do we work with these tools as an enablement to the work that we do. Because the context, the credibility, the years of experience and understanding, um, the human judgment, those are all still going to be really important to the work that we deliver every day. And so how do you combine those skills with the best technology to drive the right solutions for our clients? And that will be the test. And I would imagine that's going to be the test in every industry. Although I'm curious to see how AI is going to disrupt the dental field. Maybe it'll make it hurt less. I don't know, that would be nice. But it'll be interesting to see how it disrupts pretty much every industry.

Speaker B: One of the other things that you mentioned is entry level talent and new grads. And this is an area where, um, I think you can make the argument they're being hit the hardest. And there's this growing trend now where when, uh, in commencement ceremonies, when speakers are getting up there giving their speeches to the graduating class, they're now getting booed whenever the commencement speaker mentions AI. This happened with Eric Schmidt recently, the former CEO of Google. All these commencement speakers are somehow touching AI. You know, congratulations, welcome into this new AI world. Learn how to use these tools. And then all the college grads are boo, boo. Like there's this very strong backlash against it, uh, especially for entry level and new grads. So for PwC, how is this impacting your new grad and talent pipeline? Because for so many companies out there, it seems to be that the argument they're making is, well, do we really need new grads now when we can use AI tools that can kind of do what the new grads do? So how are you thinking through new grads, entry level talent and that relationship with AI?

Speaker C: Yeah, it's, I mean, my goodness, imagine sitting at a college graduation and the world has changed so dramatically from when you enter that university four years prior. And you thought, I'm picking the major that is going to guarantee, you know, a lifetime of employment and now the way that that work gets done is disrupted. So I'm not surprised that there are some strong reactions. We still continue to bring in an incredible group of, uh, new grads into our business every day. And we have diversified that group. There are more engineers and more technologists in that group than perhaps there was two years ago. And so the diversity of that group and the types of skills that they're bringing in has changed. And I think so more, you m

Speaker B: said more technologists and more people with tech skills. Because, um, the myth, the general perception is that you don't need people with tech skills anymore. You don't need software engineers because you're going to have AI for that. But you're saying you're actually bringing in more people who have those skills.

Speaker C: We are. Because you're still going to need these incredible builders and translators of technology to help drive our business. And so we are bringing in more people with tech skills. Their, their careers will look different here than they perhaps might look in a traditional technology organization. And in fact, I spent last week out in Seattle. We have a program that we just launched called engineer your career where we, we brought in a group of several hundred students who had just completed their second year of college to introduce them for a week to what a career at PwC would look like for a technologist. And it was incredible. And these, these young, bright minds were in the room and we had all of our tech, so many of our technologists in the room. And we talked about the unique role that they would play in organizations like ours where they are builders and they are orchestrators and they are advisors and they are this incredible connection point that's sort of bringing all of this to life. So we have broadened the, the talent that we are attracting from an entry level perspective. But we do know that the world is changing, Jacob, and it's changing in a way that we can't even fully predict as we sit here today. So one of the things that we did, um, last year through Pilot and that will launch this year full on, is create a program called Associate Discovery. And the whole point of our Associate Discovery program is that we need to create a more agile workforce of human beings who two plus years from now will be deployable where we need them most. And it may not be in the part of the business that they're walking into this summer as they start to join our firm. Um, it may not be doing exactly the type of work that they envisioned when they entered university four years ago, but we will give them the training and the tools that they need to be deployed where we need them most in two and a half years. So this Associate Discovery program is a really important way that we are addressing the changing environment. Because I agree with you, we suspect a lot of the work that our people did every single day at the bottom of our pyramid will be disrupted by AI. Uh, so how do we make sure that they're getting those skills in a different way? How do we enable them to do higher order work earlier in their careers than they would have? How do we diversify the experiences that they're getting in the firm so that they can play in those growth areas that maybe will have less AI disruption in the future? And how do we make sure that we're giving them the skills and the capabilities that will help them both in PwC and outside of PwC. And we are doing that through a program called Associate Discovery. So that is the way in which we are addressing what we think will be an evolving landscape for entry level talent over the next couple of years. And look, I had a great time last week at our Engineer your Career program. And I challenged the team to say, what would this look like for liberal arts? I think the incredible world that we are moving into is expertise will always be important, but it can be developed in different ways with different disciplines. And so the traditional people that you would think of as joining, um, a firm that does audit and advisory and tax work, we're still going to need those people. Absolutely. But we're also going to want those engineers and those data scientists and those philosophy majors. And we're going to bring this incredible set of skills together to build the workforce of the future. And what that looks like today, I couldn't tell you with a degree of certainty. But what I can tell you is we are taking steps to make sure that we have the agile workforce that we need to deliver against it two plus years from now. So I'm excited for the future. I understand that it is scary, um, for new grads. I have a daughter who graduated college a year ago. I have another daughter who is just a rising junior in college. So I understand the anxiety of, of people who are like, building these careers with a level of uncertainty. And I understand the anxiety of the parents who are funding the building of these educations with a level of uncertainty. But I also think the future is just incredibly bright and exciting because we are going to bring a multitude of different talents to bear for so many important challenges in the corporate sector, in the private, you know, in the private sector, in the public sector, in the nonprofit sector. And I'm excited to see what the world looks like a couple of years from now.

Speaker B: Yeah, I'm Very much on the opportunity side. Because as much as people talk about the dangers of AI and the doom and gloomism of AI, there's a lot of opportunity to build and create so much to build. Anything that you want? Pretty much. Um, I'm really curious, maybe one more question on the talent side. Have you changed the way that you train, um, new grads when they enter PwC, or have you changed the things that you're looking for at all from new grads? Uh, compared to, I don't know, maybe five years ago?

Speaker C: Absolutely. So we are, this year as we go back to campus and go into the recruiting market, we'll be conducting, um, a human skills assessment for the first time. And so we're thinking about for the first time. So we've had different assessments that we've used and we have case studies that we use in different parts of our business. And so we've, we've done that for a long time. This year we are rolling out a human skills assessment because we really do believe that our edge is always going to be human at PwC. And so attracting people with the right level of curiosity and innovation and critical thinking skills and empathy, getting those people attracted to our firm early is going to give us a strategic advantage. So we want to understand human skills. The other thing we're doing is as soon as our people join us from their first, um, orientation with the firm, they are going to immersive AI and human skills training where we are teaching them human skills, we are teaching them AI skills, we are teaching them how to use AI responsibly in our environments, which will feel different than their academic environments. And academic institutions are at different levels in terms of embedding AI into their curriculum and how it's being used on university campuses. That's another one of those things that will level out in the next few years. But right now, um, there are some differences there. So we need to level set and make sure that we are teaching our people how to use AI skills in our environment. But just as importantly, so you're going to have your, if you're a tax first year associate, you're going to have your technical training, you're going to have your AI training and you're going to have your human skills training and you're going to have all of that together. That is new and that is different in terms of the way that we are bringing new entrants into our firm.

Speaker B: Can you quickly share the skills assessment, like what that is, what are you looking for, how is it administered? Just Any more context on that?

Speaker C: Mhm. So we will use different scenarios and sort of case studies with people and give them, um, simulations to help us assess their critical reasoning skills, to help us assess their empathy skills, their curiosity and their innovation skills. And there are a group of scientists who are developing this, who are way smarter than me, who have developed this for us and will roll it out. Um, but it is designed to help us understand the baseline for how people are entering the firm and to understand how these human skills are still being developed on campuses. I believe that one of the things that we're, one of the reasons we are so focused on human skills is when you think about skills like critical judgment or you think about skills like deductive reasoning, these are skills that we all exercised every day in our everyday work. These are skills that we have to exercise with intention in the age of AI, where the technology is going to step in and actually do some of that work for us. So what we're trying to do is assess are these skills still being developed at the university level, at the levels we need, and if not, what do we need to do to fill the gap when we get here? So that's the purpose, the, of, of some of that assessment.

Speaker B: Very cool. Um, this has been a fascinating conversation. I'm so excited to follow along and see some of the new things that you're going to be rolling out. Um, are you going to be sharing any of this? Are you guys going to be documenting the journey, the programs, anything like that?

Speaker C: We will be. We're actually in conversations with a couple of universities about case studies and thinking about our kind of AI human skills journey. So we're, we're working on that, um, and just thinking about what is the right level of thought leadership to help share what we're doing and what we're experimenting with and what we're learning, quite frankly, because we are moving quickly, we are learning quickly. And as I said, as we talked about earlier, the nature of the work that we do puts us in this privileged position of experimenting early and failing early. So we're going to experiment, we're going to fail, we're going to adjust. Um, that's a privilege that we have because of the work that we do and we want to make sure that we're sharing that in the right way. So absolutely, you will see more of that to come. Um, as we continue to innovate and grow across our firm.

Speaker B: Where can people go to learn more about you? PwC, anything that you want to mention for people to check out? LinkedIn uh, if you're active on there,

Speaker C: anything at all, you can always find me on LinkedIn. You can follow me on LinkedIn. I'm pretty active. We've written a blog about human skills and the edge of humanity as we think about AI. So you, you could follow along. I invite people to follow our PWC throughout all of our socials and also our website. We're always putting new and innovative thought leadership out there around AI, around the future of work, around the incredible importance of human skills. So please follow along. I look forward to hearing from people and understanding how it's all landing for them.

Speaker B: Yolanda, thank you so much for taking time out of your day. I really appreciate it and it was very interesting to learn more about how you're approaching all this stuff with AI. So thank you.

Speaker A: Thank you.

Speaker C: Thank you, Jacob.

Speaker B: All right.

Speaker D: Marketing is hard, but I'll tell you a little secret. It doesn't have to be. Let me point something out. You're listening to, uh, a podcast right now and it's great. You love the host. You seek it out and download it. You listen to it while driving, working out, cooking, even going to the bathroom. Podcasts are a pretty close companion. And this is a podcast ad. Did I get your attention? You can reach great listeners like yourself with podcast advertising from Libsyn Ads. Choose from hundreds of top podcasts offering host endorsements or run a pre produced ad like this one across thousands of shows. To reach your target audience in their favorite podcasts with Libsynads, go to libsynads. Com, that's L, I B S Y N Ads. Com. Today.