672. Robot-Proof Yourself: Navigating AI's Impact on Work and Life with Vivienne Ming

Speaker A: Professors fm.

Speaker B: You're listening to the Unsiloed podcast with Greg LeBlanc. Produced by University FM. Unsiloed is a series of interdisciplinary conversations that inspire new ways of thinking about our world. So wherever you are, enjoy today's episode. And here's your host, Greg LeBlanc.

Speaker C: Welcome to Unsilable High Load. This is Greg LeBlanc, and I'm here today with Vivian Ming, who is the executive chair at a nonprofit called Human Trust and also does some work here at UC Berkeley and will soon be at Insead and is also the author of this wonderful book called Robot Proof. When Machines have All the Answers, Build Better people. Welcome, Vivian.

Speaker A: It's a pleasure to be here. It's always fun to chat.

Speaker C: Well, I should have mentioned also that you've been involved in quite a few startups in the HR tech and ed tech space. I'm never really sure exactly where the boundary is between those two things.

Speaker A: I remember I was, I have yet to learn my lesson. It's always a horrible experience, but it's addictive. So, yeah, I've done a couple of ed tech startups, plus a lot of education, nonprofit, um, work. Probably my greatest claim to entrepreneurial notoriety was as the chief scientist of a company called Guild, just launched my 13th organization called Possibility Sciences, where we're literally applying AI to science and innovation itself. I like the speed of that, I like the discovery of academia, and I like being able to work on the things that actually get me out of bed in the morning on the philanthropic side. So why choose?

Speaker C: Well, I mean, look, you've been on the inside of hiring algorithms, right? So using AI in hiring and in education. And, um, the subject matter of your new book is really less about using AI in the hiring process and understanding how AI is going to change the workforce. And I mean, this is probably one of the biggest discussion points and debates out there. I just concluded yesterday a course on AI transformation for organizations with my MBA students. And, you know, this is something that they're interested in not only because they're interested in their careers and figuring out how they're going to be relevant going forward, but also as managers, right. Trying to figure out exactly what the composition of the workforce is going to look like, what the interface is going to look like, what the division of labor is going to look like between agents and humans. And I think that the answer is we don't really know. But there's a lot of theories floating around. And I think you're targeting a lot of these theories because a lot of them seem Kind of, I don't know, they seem kind of lazy. I mean, they seem to be built on our, uh, prior experiences with say, the first Industrial Revolution, which is a very, very different thing. And so, I mean, I guess the first question would be, you know, why aren't we giving more thought to this? Because this seems like a pretty important question right now.

Speaker A: I mean, I spend a lot of time in the book using this phrase, the Lazy Myths. We have this phenomenally complex technology. Actually, weirdly, AI, uh, I'll use that as a sort of product name as opposed to machine learning, which is what jerks like me work on under the hood. So we have maybe the most complex thing humanity has ever invented and then so many just ignore. It's interacting with actually the most complex thing that exists in the universe, which is the human brain. And then we have another layer on top of it which is culture and management. And these three things, each of them is essentially already an unpredictable chaotic system. Forgive me for nerding out with you now. They interact in even more unpredictable ways. And the laziness with which so many self appointed sages have said, oh, don't worry about it, it's just like the Industrial Revolution, or haven't you ever heard of the Gervons Paradox? Or it'll do all the boring work, you can do all the fun stuff with the last one being, I think, the great sales pitch of agentic models. I want to start off by saying I get to be everybody's enemy here. I am scornful equally of the AI utopianess and of the dystopianess. It's not going to destroy every job, it's not going to end the world. No, it's not a scammy sales pitch. For 30 years I've gotten to build an AI that can predict manic episodes and bipolar sufferers, can predict my own son's blood glucose levels into the future to help treat his diabetes, reunite orphans, the refugees with their extended family members. There's something real here, something that no human can do without this immensely powerful tool. But we get caught up in these lazy myths when it is such a hard story. So I think part of it is just, I'll be a little honest here. So many of the people sharing these visions don't actually come from a background of having to understand the complexity of it all. You know, computer scientists, despite the name, aren't scientists. They're engineers for the most part. And they are the biggest publishers of, uh, both research and blog posts about AI. The next biggest are Business school professors. I also don't mean to knock them. I am now soon to become one of you, but for the most part also not scientists. And what neither of them are are cognitive scientists, people that actually study human intelligence as well. So we're one of the biggest lazy myths here is I can imagine a world in which AI makes all of productivity better and everyone is freed up to be artists and scientists or whatever the hell they want to be. Not in the real world. In the real world, for one thing, humans are all different. And so when we interact with AI, or for that matter, just social media or the Internet, turns out we all behave wildly differently. Some of us are net positive, most of us unfortunately are, uh, net negative. Understanding that messiness is where the real stories are. And yeah, because it's messy and chaotic, it's kind of fundamentally unpredictable. But what we can see is trends. The trends look really bad. But there are also futures in which things are much more inclusive. And, and I'm going to argue that those more inclusive futures are actually better from a bottom line standpoint. The, uh, thinking of AI as an army of robotic minions to do whatever you want is actually really limiting. It's really limiting in terms of capabilities, in terms of innovation, and even in terms of productivity. And rather than just guess, I decided to run some experiments to see what actually points us towards a better future. It's just the results of those experiments say the better future is effortful rather than lazy. And I'm going to be honest, the startup world is built on lazy. It's built on hyper efficiency and giving people what they want and not what they need. And we've productized AI in exactly that sort of genre of lazy product development. Because if you want that hockey stick, it's gotta be lazy, it's gotta be immediate, and it's gotta satisfy people's sort of shallowest self itch. But eventually that's really gonna come to bite us on the ass. The companies that figure this out early, the countries, the policymakers I think are gonna reap astonishing benefits. But you know, the honest answer is those benefits come with costs and right now no one's talking about paying them.

Speaker C: Well, look, I mean, the kind of conventional view that you read in the media, the ones designed to get the clicks and so forth, is that, oh my gosh, these systems can do everything better than us, and so we've got nothing to do and so we're just going to hang out and watch them do all the work. Right? And of course an economist would come back and say no, no, no, like it's about comparative advantage. And there's going to be some things that these tools are going to be comparatively better at and there's going to be some things that we as humans are going to be comparatively better at. And that's going to be the new division of labor. And we're going to have, you know, complementarity and we're going to have some exchange and we're going to have some, you know, cyborgic interfaces between us. And so what is the difference between the view on the one hand that this is a substitute for us and the other view that it is a compliment to us?

Speaker A: Yeah, you know that I think there's two distinct issues to address in this space and one is this one of complementarity and substitutability or sometimes complementarity as augmentation versus automation is another way of thinking about it in less pure economic terms. But there's also other dimensions of really understanding why there's some intuitive appeal to a sort of shallow approach to AI development. Because we keep measuring AI capability using benchmarks in which the AI is doing stuff entirely on its own. So essentially we've come up with the equivalent of ah, the Scantron test for AI. There are explicitly right and wrong answers. They can be very complicated. Humans can really struggle with these tests bluntly by design, so do the AIs. But to be clear, they are what I'm going to call well posed problems by design. The way for example people doing reinforcement learning, AI modeling. So this is the original domain of organizations like DeepMind and AlphaFold and AlphaGo where you just turn it loose in a simulated world and for example let it play go against itself and in a single night like 186 years of human gameplay. God, it's not shocking it can beat the best human players after something like that. Maybe what is shocking is it's not godlike in its ability. It doesn't just trounce us despite our limitations. So we've historically in computer science built these benchmark systems that have explicitly right and wrong answers and we test AI against it. Well, guess what? That's not the real world. A benchmark on cancer detection in medical imaging is not medical diagnostics by real world doctors using a variety of X rays in a variety of places in the world. So those benchmarks turn out to be informative, but not nearly as informative as people seem to think they are. And the reason for that is as opposed to well posed problems where we understand the question, we know how to get the right answer. And frankly, modern AI simply is superhuman and its capabilities. So you really have to look at these capabilities as if the world is made up of nothing but well posed problems. Yeah, we're pretty superfluous, but it isn't. In fact the most interesting problems that feel rare to us but in almost the entire game are the ill posed problems. These are the things where forget right answers, we don't even know what the questions are. And it may seem sort of trivially, almost axiomatic to say that while the space of well posed problems is vast, everything humanity has figured out, like play around with Gemini or GPT or any model you want and be proud, what it can produce is a testament to what humanity has discovered about the world. And sometimes it produces ugly things. And that's a reflection of us too. But the set of things we do not yet know is infinite. Our relative advantage is in ill posed problems. This is what my experimental research is about, how we deal with uncertainty in the unknown. And let's be clear, humans are terrible at this. We are genuinely atrocious at ill posed problems, but frankly, we are the only game in town. So bad as we are, we have relative advantage here. And I think some of the genuinely best thinkers in this space, like Nobel prize winner Darren Acemoglu and his frequent publishing colleague David Autor, have had a huge influence on my thinking. Because what they show is what you were talking about, complementarity and substitution is actually very heterogeneous. By which I mean different people with different human capital capabilities can be both complementary and ripe for substitution. And the question is how ill posed or well posed a problem is. And it turns out as human capital accumulates, not traditional university derived knowledge, but human capital in the form of curiosity, working memory span, perspective taking. As that accumulates, we become increasingly complimentary to modern AI. But down on the other end, the substitutability, or a different kind of complementarity, is it's no longer worthwhile to build an AI, to wash dishes or to pick lettuce in the field. It's expensive to build robots, literal embodied robots. So you know what? Humans are better at that. And what Otter and Osimogo have shown is that the demand for labor is profoundly U shaped, massively spiked at the highest and lowest end and then negative demand change, so slowing demand in the middle. And what that's really showing us, I think is two separate things. Huge complementarity for, uh, the ability to solve ill posed problems and unfortunately a growing complementarity for your Ability to do the boring things that people need but nobody actually wants to do. And everything in between is increasingly automatized.

Speaker C: Well, so I think you construct sort of a two dimensional view of human capabilities. Right. So there's the one that most people like to talk about which has to do with skill levels or education levels. Right. So people with very little education, people with high degrees of education. So maybe you're high school dropout and your PhD. But then there's an entirely different dimension which aligns, uh, with what you're discussing, which I guess we could think of as routine versus creative or something, which is.

Speaker A: Those are terms I use in the book just to offer some real intuitive grounding of, uh, what we're talking about here.

Speaker C: And. Yeah, but most people, I think, when they think about this vertical. You know, this is maybe why all of the folks were booing the speakers at the recent commencements. Right. Cause they think of themselves. You know, I'm a college graduate. I spent all these years studying. I've memorized all this stuff. I've gotten A's in all my classes and so forth. And now I'm like the weavers of the 1800s. I'm, um, the spinners and the machines are coming for me. And so I'm in big trouble.

Speaker A: Right.

Speaker C: And that's sort of the narrative that while the automation in the first industrial revolution got rid of a lot of these sorts of artisanal workers, now it's coming after the knowledge workers.

Speaker A: Yeah. You know that all of those commencement speakers on some level deserved to get booed. The thinking behind it, at least if you're going, if you're really going to pin it back against like the Jacquard loom completely disrupting weaving. Yes. The mechanism of automated textiles disrupted the textile industry, but. But it gave rise to the fashion industry, which is vastly bigger. Well, the yada yada yada part of that is captured in a, uh, message from a British diplomat who said in something like 1870, the plains of India are bleached white with the bones of Indian weavers. Other than agriculture, it was the largest industry on the planet because almost everybody did it everywhere.

Speaker C: They're gonna have the bleached bones of accountants and lawyers. We're just gonna have their bones out there in the streets of New York.

Speaker A: So half of this is sort of trivializing the reality that skilled artisans lost their job and were replaced by little kids whose tiny fingers were much better at putting the bobbins on these automated looms. Yes. Eventually it transformed into these new industries but it took time, even in the British Midlands, much less around the world. So one thing, we live in a modern economy. We shouldn't be so dismissive of these realities that yes, the Gervons paradox exists. I agree with Gervons. There is this finding that as the cost of doing a certain kind of labor drops, seemingly erasing that, uh, career, but demand increases. So really you create a change in the career, but it also misses the fact that usually that means the people that were originally doing the work are out. A generation or more later a whole new group comes in. When you get booed, you have to realize you have put those people out of work. What do they care that somebody else, somewhere down the line we'll have a different kind of job? But I also want to point at the booers because I think that there's a lack of introspection here. Did you just go to university because you thought it would give you a high paying job? Was that the point of it? Because I will tell you if it is, maybe go do something else. Did you do it to scratch your ego? I guess because you could get a high paying job. So I again really look at baseline human capital accumulation and almost all positive life outcomes from health to economics to well being are not defined by whether you know how to factorize a polynomial or whether you went to Harvard or Stanford. They are almost always driven by much deeper and I loathe the term soft skills, but I've already run through a short list of some of these things, these deeper human capital qualities that on top of which you can start to layer technical skills like mathematics or chemistry, but those do you no good. So viewing your college education simply as a license to earn money I think also deserves a lot of self skepticism. So we need to really transform work, but we also need to transform people's relationships with work because we simply don't run a factory line economy anymore. I don't need you to be a very sophisticated cog where you're given orders and you execute them and only a few people are smart enough like you to execute those orders. And pretty much everyone's job. Now if I may be so self elevating is my job. People bring you unknown problems, you got to figure them out and I think that's terrifying. And no one's educating the next generation workforce on how to do it. So I boo both sides though with much more scorn for the wealthy because it's so self serving of them to say, you know, my billion dollars in OpenAI stock has Nothing to do with what I'm saying here today, but take my word for it.

Speaker C: It' but look, I mean, you're going to talk about the different skills that you need to succeed in this future world full of AI, but it seems like your work at Guild highlighted that those skills actually were already the skills that helped you to succeed. So in other words, this is actually a story of continuity. It's not so much a radical disruption, it's going to force us to acknowledge, I guess, something that we didn't always acknowledge. I found it fascinating that when you talked about the recruiters at a lot of these companies that these recruiters, they simply. I mean, I have a couple friends who started an HR analytics company and they also discovered that the criteria that the HR managers were using and the recruiters were using did not line up with performance. And yet many of them just kept doing it, you know, like, hey, let me see your CV and let me see what school you went to and so forth. And so is there really as much of a discontinuity as people think or have these skills, these soft skills more or less always been the ones that help you to succeed in the knowledge economy.

Speaker A: So obviously I'm going to agree with what you're saying, since it's right out of my book, is AI is a forcing function here. It hasn't changed the underlying reality that almost certainly what I'm going to call meta learning, your ability to learn how to learn, or another term I really like is foundation skills. That term comes from the idea that, yeah, it does actually there is truly value in going to university or knowing how to write code or do complex math, but only once you have this foundation. So there's some great research there showing that without these foundation skills, those higher level skills don't do you any good. But we've seen skipped that in our sort of global education policy. Everyone should go to university, everyone needs to learn how to program, everyone needs STEM skills. I mean, hey, I'm a scientist who not only went to university, but ended up becoming a professor. Uh, I love this stuff, but I'm saying there's no evidence that that does you any good without foundation skills, without meta learning. Now, over the last 200, 300 years, we have had this period in which there was real value in a human being simply being able to add up a whole bunch of numbers on a spreadsheet and produce an answer to a highly standardized question, because nothing else could do that.

Speaker C: So you're saying it's an interaction term, Right? So you know, you can certainly have a career based on skill acquisition, but if you have these meta learning skills that uh, will sort of superpower your ability to.

Speaker B: Yeah.

Speaker A: Ah, now that has changed a little bit in recent years, and I'll get to that in a moment. But there's definitely an interaction I wouldn't want anyone to think. I'm saying it does you no good to know how to do a thing. But I guess what I'm building towards is nowadays the only real value is knowing why to do a thing. And so when we look at all of this, there was almost a tongue in cheek paper when for a little while people were calling this 21st century skills. Someone did an actual natural, uh, language processing analysis of the writings of famous like 16th century figures and found as it turns out, 21st century skills were really important in the 16th century. It's just most people were living lives of subsistence and so it didn't play out that much in most people's lives. And then that was replaced with the Industrial revolution. And again, it wasn't a fundamental requirement to simply feed your family that you had this. Now we're finally in a world where there is this thing that does well posed problems better than we do a lot of what traditional education was meant to do. In fact, nowadays you might argue traditional education just turns us, uh, into worse robots. Our ability to answer well posed questions simply does not compare to what I can get virtually for free out of my phone, even for really complicated tasks. And yet we're still investing in humans as though AIs can't do it better. And so if these qualities, these foundational skills have always been predictive of human life outcomes, the fundamentals, health, wealth and happiness, now we have the chance to let go of the idea that your job is to pull a factory lever, however intellectually complicated, again and again and again for 40 years, and then retire and finally do the thing you actually care about. And the real question then becomes, well, how many people can we kind of pull out of this old economy and into this new one? And this is again where the question of the real variability and heterogeneity in humanity becomes more than a nerdy academic question for someone like me. It becomes fundamental. Because the answer to the current workforce is way more than currently existing, but probably nowhere near a majority of workers. And then the answer for the next generation is we can transform work and change the world they're coming into, but it requires almost completely changing our sort of global concept of what education is. I don't see either of those happening right now. Not that it's impossible to do it. I'm just saying there's no trend data suggesting massive dramatic changes in either education or current workforce. We have to make those changes. Or again, my point isn't that the world's about to end, but what I'm saying is we're headed towards a massive undervaluation of human potential and it's going to cascade. My biggest fear of what AI is bringing us isn't Skynet. It isn't the world ending. It is angry young men, uh, experiencing 30, 40, 50% unemployment rates and having nothing to do. And no society survives that.

Speaker C: Well, to be clear, I mean, to have a job where you are utilizing these metacognitive skills, even in a very lucrative position, that's not necessarily the same as kind of doing what you want to do. I mean, I, uh, bet you know there's going to be a lot of people who want to be poets who are going to be working as management consultants. Right. You know, it's stuff. So, you know, I think there's this fantasy that, oh, we get to do all the fun things in this future world. You know, there still has to be demand for what it is that you're doing if you want to make a living at it. Right?

Speaker A: Yeah. So sometimes we end up with, oh, uh, well, not everyone can be a doctor and not everyone can be a CEO. And I get that argument again, I don't know that everyone actually is ready to sort of fully make a transition into this world in which kind of creative labor dominates. But we need to rethink the human value add into our economy. It's much more complicated than I need a standardized answer to this problem because yeah, stuff like that really should be automated. Not because humans don't have value, but because maybe we don't really want humans just being a cog in the machine. So we have this idea that the only point of a human is to answer a well posed question. And if an AI can do it better than why are the humans there? Or M, as you note, you know, we'll just pay everyone a universal basic income and they'll go do whatever they want to do. As I detail in my book, there are good reasons to believe that that's not true for the majority of people. If you pay them a ubi, they don't suddenly magically become more creative. It doesn't change their life trajectories. The truly unfortunate reality is those play halo and hang out. And of course an economist would also say, but if everyone gets paid to then pay their rent, then rents will just go up. So I am not a fan of ubi. My politics are pretty progressive, but I run a philanthropy and I pay for everything in the work that we do. So I have developed this belief that the only thing that matters is what works. And UBI simply is not a solution to any of the problems that are sitting in front of us today. So we need to rethink what human value is. So one of my arguments here is of the existing workforce. It would be delusional to think everyone is going to switch over and enter the creative economy. We made similar promises with globalization and those promises were not kept. Is it any surprise that a bunch of angry voters in the US and Europe that used to have sort of reliable, decent jobs are now voting for people promising to bring back the past? You promise to turn a coal miner into a Google employed software developer is one, a complete fiction and two, whether people even meant to keep that promise, they didn't because you can't build the underlying foundation skills in a six week job retraining program. So these are part of the kind of lazy myths of uh, the future of work that I wrote about in the book. And yet they keep coming back because it's so much easier to sell lazy than to sell hard. And hard is totally changing how we raise the next generation. Hard is, as I explore in my recent experiment, building AI models that actually pull more from humans rather than push towards them. And finally, hard is if there really are productivity boons from ubiquitous AI in our economy and we are seeing that just not where the lazy predictors were expecting it. It is in with elite workers with high human capital, as my experiments have supported, that we need something real and meaningful for a huge chunk of the workforce to do today. And washing dishes is not it. You know, you're looking at taking the AI dividend and rebuilding every bridge in the United States. Give people real meaningful work if they're not ready to transition to the creative economy. But I think there are many more people that can do this. So now I feel like it is the time to jump into this experiment I keep alluding to. But I'll just say if a not unreasonable estimate of the current creative labor pool is maybe 1% of existing workers, 1% more is doubling that. That's a huge change and research would suggest it's genuinely achievable. My dream is more like 10 or 20%, which is a massive like global effort. But I, uh, think it is a doable thing that you can appreciate that's kind of moonshotting. This is a real moment of, uh, saying we are preparing for the future. How many billions and billions got spent preparing for Y2K or now is being spent, you know, forward quantum encrypting every password because maybe it will get hacked by quantum computing? Why are we not putting similar efforts towards unlocking human capital?

Speaker C: Well, look, I mean, this type of human capital that you're talking about, right? I mean, there have been people in the world of education who have been discussing this for a long time, right? And they've been talking about how the educational system, the parenting system, needs to orient itself towards providing these types of skills. But I mean, I'm just continuously surprised because I've taught at many of the top universities in the world world, and I still encounter students who think that success means acquiring these knowledge bases and these skills, uh, rather than these other more metacognitive skills. I mean, why is that? And you know, what's been the difficulty in reorienting the educational. I mean, I feel like I got a great education. I mean, I went to Montessori school. You know, I had a Jesuit school. And so, you know, by the time I got to college, I didn't care about grades, I didn't care about fulfilling requirements. You know, I was just having a field day learning. But I think I was the exception. So why is it that we can't do this? I mean, what are the obstacles? Is it just simply that it's. I mean, it seems like it's a more difficult thing to teach in a structured way, and it's more difficult to evaluate and to measure progress in.

Speaker A: It's certainly more difficult. I think that's the right way of thinking about it. And using this term soft skills, what we have convinced ourselves is something like they are impossible to measure. There's some sort of ephemeral quality of humanity, a grace granted us by whatever mysticism you subscribe to. I'm a computational neuroscientist. I simply don't look in humans in those terms. It is hard to measure. There is real variability in populations across this. It's also a product of all sorts of different underlying factors, from genetics to non genetic biological factors, to development to home life to context. We are the world's messiest democracies, our brains casting millions of votes at any given moment, of which we only actually count a few. And then we build this post hoc story about us. Layered on top of the complexities of individuals is the complexities of, of managers and policymakers whose whole careers over the last 200 years has been how do we reduce uncertainty? What is management? It is your ability to define a highly reproducible process and fit people and capital into that process to generate a predictable outcome. I have just argued that the unique human value in the modern economy is our ability to deal with the unknown and uncertainty. So what good does traditional management do if human value is throw them into the unknown? Find wherever the parts of your organization are that are out in the fog of war. And that's where the humans go. Everything you fully understand, that's where the machines go. This is what we've been up against is it's so much easier to say, at some point someone is going to need to analyze a spreadsheet. So let's start by trying to teach every kid how to reproducibly add numbers and then do slightly more complicated mathematical operations up until we can slot them into being a financial analyst. And hey, it turns out in the 90s a talented financial analyst in Mumbai is just as good as a talented financial analyst in London. But they cost half as much. So we're going to send all those jobs there. And now it turns out that a GPT driven analyst is even better and phenomenally cheaper than the human in Mumbai was. So now we're going to onshore it back, but automate it. And so this process has been going on because dealing with uncertainty is immensely challenging. Like I said earlier, we're the best that exists at it and we're terrible. Well, guess what? The managers and policymakers are evidencing just how terrible we are at dealing with uncertainty. So they want to magically make it go away. You mentioned Montessori. I don't care. Montessori, Waldorf, whatever your student centered education system is you want, that's where our global education system has to go. So this means project based, collaborative, student centered, intergenerational. A great study was done and I think published in the Proceedings of the National Academy of Science showing that a Montessori preschool in Michigan, these young kids were there getting prepared when they entered the primary Michigan education system. They really didn't look dramatically different than the other entering students. But as you track them across their academic careers and beyond, they had much better outcomes. The reason I bring this up is because that was a public Montessori system, publicly funded, publicly run, better outcomes. Look at all of James Heckman's work. Nobel prize winning economist. Look at Raj Chetty's results that noted Harp, probably someday Nobel prize winner himself, if I may Wildly inflate myself in this space. Look at my research.

Speaker C: And then there were those interventions I think you cited.

Speaker A: So this is some of uh, I mean the actual interventions aren't James Heckman's, but the analysis of joint work between his group at University of Chicago and a group at UC Berkeley, sort of often notoriously called the Jamaica. The study looking at at risk kids in Kingston, Jamaica and what they found was training their parents in sort of little social emotional games, development games they could play with their kids, dramatically changed those kids outcomes. They earn 40% more, they're dramatically less likely to be incarcerated. In some very closely matched research in the states with similar at risk populations found they had lower cortisol levels. So this is a uh, sort of stress related hormone that's a uh, big predictor. Chronically elevated cortisol levels is a big predictor of poor health outcomes. So social emotional development, I. E. Meta learning foundation skills, improves incomes, decreases negative health and well being outcomes. So here we have the results. In my book I have a little subsection titled if kids were bonds, they'd be the backbone of the world economy. Because as Hackman and my own research shows, investing in those kids has shocking, shockingly high returns. The Heckman estimate is every dollar invested in a kid returns $7, you know, within 20 to 30 years. Where else, what bond can you get those kinds of returns in? So, well, the idea is M. The

Speaker C: ROI is going to go up, right? I think that's your claim is that you know, in the past, this is.

Speaker A: My claim is obviously with the complementarity of AI, the value of that rises dramatically. There's already value, I mean demonstrable. This is what we were saying earlier is this is true without AI, lower healthcare costs, lower policing, incarceration costs, higher baseline productivity. But my real argument is developing foundation skills isn't just about a better job, it's about a better life. What an amazing opportunity to take this moment in history and not just concern ourselves with whether I'm going to boo Eric Schmidt because he's doubling down on AI and I feel like I'm losing a job. But to actually see that moment as an opportunity to invest in humanity in the way almost paradoxically we always should have been. But we didn't have to. And so we didn't.

Speaker C: Well you know it's funny, I mean when I encounter my students after graduation, right from MBA program, 10 years later, 20 years later, they usually will say that the most valuable thing that they learned in business school. I've never heard anybody say that it was how to do a discounted cash flow or how to do a two sided T test. What they'll tell me is I learned about self efficacy, I learned about metacognition, I learned about all the things that you're talking about. And they're never explicitly in the curriculum, but if you design the curriculum, well, then they're implicit in the curriculum. So you wrote a couple chapters. One is on robot proofing yourself, one is about robot proofing your children, and the other was sort of, uh, about organizations and how they can foster a career path that enhances the skills of their employees. So I want to talk about robot proofing on the latter issue. Do companies really have an incentive to do this? I mean, it seems like it's more or less up to the individual because the companies, you know, if they invest in this, they may not ever see the reward because the people will just kind of, you know, move on and go to other companies and, you know, carve out a career after they leave that company.

Speaker A: You know, it's somewhat interesting how much economists and business leaders advocate for, at least in economic terms, a positive sum view of the world until it's time for them to invest in an uncertain future. At which point it's, but my employees are only going to be here for two to three years. So then we end up, we're all defectors in a prisoner's dilemma problem where we knew if we would invest in our employees and you invest in yours, then we're all going to reap benefits. And as it turns out, there's research on this. When you look at industries and regions where companies invest, even knowing that those employees are not likely to, all of them are not likely to remain employees for them for their entire career. Like, we just don't live in that world anymore. You're not bleeding blue at IBM or spending, you know, 30 years at Exxon. In the tech industry, genuinely two to three years is, there's sort of a long tail of careerists within organizations. But two to three years and then you move because actually you get that's a better way, you earn more money that way. So, uh, the interesting thing is it truly is a positive sum world within sub domains of industries and regions where companies invest in their employees. If they do it collectively, they all increase in productivity over time. They reap the benefits of have invested because the others do it as well and you all collectively go up. So we can get trapped in a prisoner's dilemma or we can look more rationally at this and see a Way forward. And you asked earlier about why we are where we are in terms of even the students themselves, as well as the teachers, the policymakers, and the business leaders being so shallow in their approach. There was a great little study. I didn't include it in my book because it only recently came out. But since curiosity is one of the things I write about as one of the foundation skills, it popped as one of the biggest predictors of what I'm calling hybrid intelligence. When you bring humans and machines together. In my research, this study came out finding it was a controlled experiment in which teachers were trained to praise interesting questions instead of right answers.

Speaker C: Yeah, I have a dumb question award that I give to my students.

Speaker A: Exactly. And you know, I always say to my students, I hate to break it to you, but there really are dumb questions. But if you don't start asking them early, you won't figure out what a good question is. And so take the hit and grow. We can get into the neuroscience of what's going on in the heads of people that are willing to be wrong publicly. Because what's fascinating is so few students do it dramatically, few, probably not uncoincidentally around that 1% creative economy level. And yet it is so strongly and obviously predictive of positive life outcomes. It turns out in the treatment condition, when they're praising the questions, students very quickly start asking more questions. And in the other condition, obviously, students are still totally focused on right answers. But then, you know, the actual manipulation here is in independently assessed curiosity. The kids in the question condition, their curiosity goes up. And then when you introduce new scientific material into the course without any prompting, they engage more deeply with it. If we have three mechanisms that we can focus on for changing the future, we haven't talked about it, but one side is the product mechanism, the AI itself. As my research shows, we're building AI optimized for autonomous benchmarks. And that's not what we need. We need AI optimized for hybrid intelligence. It's interaction with people. We can choose the people. It's what we've been talking about. We can focus on them, develop foundation skills. The most complicated and difficult, the least explored is how do you change management policy making? Because if we continue to incentivize, just give me right answers here, they are virtually for free. So what's the point of humans if that's the only thing you care about? But it shouldn't be. I'm just going to say something provocative because we don't have time to get into it in the optimally intelligent organization. The majority of people should be wrong the majority of the time. Otherwise you're not exploring enough. And we can show it in theoretical models. We can show it in experiments. Who's managing companies to encourage productive exploratory wrongness? Almost nobody.

Speaker C: Well, look, if I had had you in to speak to my class this past week, and the final question that a student asked you was, okay, I'm an MBA and I'm graduating, what kind of skills should I have been developing over these last couple years? Or what kinds of skills should I be focusing on? You know, what would they be? I mean, on top of obviously, you know, having these characteristics of curiosity and so forth, you have a couple of specific recommendations, like how you.

Speaker A: I do have very specific recommendations, but let me preface it with the following. In my research that started with analyzing 122 million people when I was the chief scientist at Guild, the company doing AI and hiring and continuing through today is. No single skill by itself is sufficiently predictive. They are sort of predictive at a population level. But my life story is proof positive. Just being smart is not enough. Uh, I spent a bunch of years in the mid-90s homeless because I thought smart was supposed to be everything. And if I wasn't successful in life, then I was a complete failure. And so I just gave up social skill all by themselves or creativity all by itself. It's not enough. What we see is its patterns of skill. And there's no fixed pattern. It's just that the skills complement each other. So I ran an experiment. I had students at Berkeley make predictions about the future. Predictions. We took off of the site Polymarket, that is more notorious nowadays. What will the price of oil be in six months? Will there be a new front in the war in Ukraine? Will there be a war in Uganda? The students were terrible at this, right? But they don't know what the price of oil is today, much less what it'll be in six months. So the very best student did worse than the worst AI. So I had four. State of the art at the time was Gemini three. We had GPT, we had a small open source Gorn model and a small open source Llama model. That last one was the worst performer, way better than the best human beings. So again, if that's all we're doing, AI wins, we're out. What's the point? I'm sure experts can do better. And in fact, on, um, Polymarket, the experts really were doing better, or at least the market was doing better than the very best AIs. By a meaningful amount. So of course to me, the interesting thing is the hybrid intelligence question, which brings me to my answer to you, which is the majority of participants in my experiment shut their brains off and simply did whatever the AI told them. If the AI said the price of oil will be $60 a barrel, that's what they typed that in. So essentially humans became a really slow copy paste function that required health insurance, which is, um, not what your students are wanting for their career, but was the majority even of very smart UC Berkeley students. But the exciting thing is what we call the cyborgs. A term that I love because I'm a sci fi nerd, but actually started with some other researchers. The interesting thing about the cyborgs is their intelligence. The hybrid intelligence they evidenced was fundamentally emergent. It was not what the human would have chosen. It was not what the AI would have chosen alone. It was different. The AI didn't do the boring stuff. The humans and the AI together explored the low level data. The humans and the AI together explored high level hypothesis generation and creative idea generation. And they didn't just do better than the best humans, they didn't just do better than the best AIs. They effectively did as well as polymarket. Uh, making 10 predictions in an hour against people that were experts betting millions of dollars. That's the most hopeful thing I've ever found in my research. What is interesting is the AI benchmarks essentially didn't predict anything about the sideboard. So now to finally answer your questions. The four most predictive variables in that experiment were fluid intelligence, curiosity, intellectual humility, and perspective taking a social skill. So the AI gave you a good enough answer. Did you keep looking for a better one? Curiosity. The AI told you you were wrong. Did you just collapse and do whatever it said? Do you ignore it? Or do you say, okay, I get that, but what about this intellectual humility. And it turns out understanding other people is hugely predictive of understanding LLMs, large language models, because they're trained on this. An important final note is knowing how to do a thing helps. I kind of predict. Uh, I brought non experts into my study, but if you actually looked at their backgrounds, people that knew something about economics did better in making economics predictions. And people in the hybrid intelligence condition, people with a little political science background did better in the predictions about wars. Knowing a thing is another important factor. This experiment only lasted an hour. I, uh, guarantee you if we ran it longer. Resilience, sense of purpose in life, self assessment. So we're looking at meta learning Metacognitive skills, emotional intelligence, social skills. By the time they're your students in your class in your 20s, cognitive development isn't as much of an issue. That's largely on a kind of set trajectory. It's a maintenance game, which is important if you're turning your brain off and letting AI do the thinking. Oh, I don't want to be you 30 years from now. Rates of Alzheimer's will almost certainly go up as a result. But you can change if you're a parent. We're talking about little kids, 100% numeracy, literacy, working memory span, fluid intelligence are still plastic and changeable. Yes, we have genetic sort of foundations that are set in our lives. There's huge amount of variation left, just peer effects. Who you spend time with actually influences these things. And something as simple as praising questions instead of praising right answers can change these qualities about people. So that would be what I'm saying to your students. And I'm going to add one last in that I'm not explicitly citing here, the courage to tell people the truth. If I had one massive nerdy hand wavy thing to solve or at least improve many of the ills of the modern world. We need to de correlate our choices, our beliefs, our media consumption, genuinely, paradoxically, so many more choices. And yet the more choices we have, the more correlated our behavior has become. And let me be blunt, I do not need you to give me the right answer, because I already have that, by the way, all of your competitors have that, the exact same amazing right answer. I need to get you to give me the answer only you would give. That is your literal economic value to this world. And if you don't have the courage to tell me that I'm wrong and to suggest something that itself might be wrong, then I, uh, can't help.

Speaker C: Well, also the courage to be told that you're wrong. And this gets to some of the

Speaker A: exercises equally true that that profound sense of the value of productive skepticism of yourself, of other people that are part of your in group, maybe a little less so of people that are part of your out group. You know, this is where we need. If the value of humans is dealing with the unknown and the uncertain, then improving the skill set of self, of introspection, of curiosity, intellectual humility. You can see why these pop. In my research, and not just mine, many others have found perspective taking and curiosity interacting with AI in really positive and complex ways.

Speaker C: Well, and you provide some specific ways that you can do this, including ways that you can use AI to help build these skills. AI isn't just going to undermine these skills. AI can actually enhance and strengthen these skills if you prompt it in the right way and provide the right persistent instructions.

Speaker A: Yeah, it was a blast coming up with this list because that's not typically. I'm sort of a nerdy scientist and I like to explore the messiness, but it seemed appropriate for a book like this. Everybody wants things they can do. And so, yes, I have some very explicit recommendations for parents, for individuals to do for themselves and for leaders. Find the one that works for you. And, uh, not just of my list, but obviously my point is think creatively. What would only you do? How would you use an AI to actually challenge yourself? And how are you challenging AI to give you a better answer.

Speaker C: Well, Vivian, thank you so much for joining me. This book is called Robot When Machines have All the Answers Build Better People. So let's do it. Thanks so much.

Speaker A: It's a pleasure.

Speaker B: Thank you for tuning in to the Unsiloed podcast produced by University fm. Um, if you enjoyed today's episode, please give us a five star rating and review in your favorite listening app. To listen to our other episodes, please visit our website at www.unsilopodcast um.com.