The Case for Merging Brains and AI to Save Humanity, w/ D. Scott Phoenix

Speaker A: Foreign.

Speaker B: Welcome to AI Curious. My name is Jeff Wilser. I'm a journalist, I'm a human and I'm curious about AI. And even that intro I always make that is in theory up ah for grabs and might need to be changed in the future. Uh, in a sense, given what today's guest is all about, who is kind of challenging in a sense, what it could in the future even mean to be human. So let me back up. Not long ago I was in Vancouver at the TED conference and one talk left the room. Shook. I think shook is a silly word, overused, but here it is applicable. Some people walked out muttering, black mirror. You heard black mirror a lot. And at the AI dinner that night, it was a lightning rod of conversation. The talk was by Scott Phoenix, who is now a partner at the venture fund 50 years. And before you kind of write Scott off as a futurist with a wild idea, you should know some of his credentials. Scott spent over a decade building one of the more serious AI companies around the Curious, backed by Musk, Bezos, Zuckerberg and so on. Eventually absorbed into Google. He is the guy who helped teach machines to beat Captchas. So when he says we can't control AI, that's not just sci fi, it's a warning from someone who spent 10 years inside the machine. Now, Scott's thesis comes in three planks. The first is we cannot stop AI we. Then second, we cannot control AI and um, third. So to stay at the center of our own civilization, we have to become

Speaker A: more than we are, starting with a

Speaker B: literal merger of our brains and AI. Now to be clear, I am not giving necessarily a full throated endorsement of this idea and parts of it of course, I find unsettling. But this is exactly the kind of big, uncomfortable idea this show exists to

Speaker A: take seriously and to grapple with.

Speaker B: No hype, no doom, just real curiosity about where this all goes. So we get into all of it. Whether his plan is the Borg and and of course the Borg comes to mind. Scott says, no, it's not that. Or is it something stranger? We get into what a human brain merger of AI could actually look like. And also the genuinely mind bending stuff. Sharing consciousness with the people you love,

Speaker A: with your dogs, with dolphins.

Speaker B: We get into even backing up your

Speaker A: minds and why that might be the

Speaker B: only way we ever reach the stars. Now, at one point I pushed Scott on how wild it seemed to, as I put it, surrender our brains to AI his answer was interesting and said, jeff, people are um, I'm paraphrasing A bit. But Jeff, people are already surrendering their brains to AI right now, and I am the resistance. So with that, please enjoy my conversation with D. Scott Phoenix. Scott, welcome to AI Curious.

Speaker C: Thanks for having me, Jeff. It's great to be here.

Speaker B: I think it's fair to say that your TED talk, which I was there

Speaker A: for, was one of the, um, most fascinating talks there also I think generated the most buzz. A lot of folks were had a lot of questions. I think, as you know, a lot of folks like, whoa, what the hell does this mean we're going? I heard people say black mirror dystopia. It was one of the white mirror.

Speaker C: But yeah, I think there's elements of both.

Speaker A: Exactly.

Speaker B: So let's. And you also, for listeners, you had

Speaker A: an off the record session, um, that you dove into more nuance. And in that session, if you're comfortable with me sharing this, you had a distillation of kind of three planks of your thesis.

Speaker C: Let's start there.

Speaker A: Can you share what your three core tenants are? And then we'll kind of get into the implications of all of this.

Speaker C: Yeah, I think it's fundamentally about thinking about what degrees of freedom do we have given that we're entering a world where were careening towards having AGI And I think we have a couple constraints. We can't stop AI. I think that's unlikely to work because, uh, so many participants are racing towards it and it's something anyone with a computer can build. We can't control AI because the idea of a smarter thing obeying a dumber thing forever is not a stable equilibrium. And in order to thrive and have a human future, when we're humans remain at the center, we need to do something that puts us there, that keeps us competitive with AI or superior to it, so that we can remain in charge of our own civilization. And so those are basically the three chunks. We can't stop AI, we can't control AI. And so we must become something more than we are now.

Speaker A: Okay. And I have a thousand questions on each of those three things. But first, can you share kind of more plainly what step three means? What does kind of becoming greater than ourselves mean? Um, in kind of layman's terms.

Speaker C: So I think the, the, there's a couple paths to this, and I only covered one in the TED talk, which was specifically about brain computer interfaces. Uh, but there's also enhancing our biology. So I'll talk, I'll take them in separate, in separate paths, but they really connect together. So brain computer interfaces are about, uh, Devices that can read and write to the brain. And so this is everything from, uh, the monitor you're looking in right now, which can only write into your brain, to very basic, uh, headsets that can, you know, do some electrical activity through your skin, to more complicated devices that use ultrasound, all the way up to invasive brain implants like neuralink and what Science Corporation's building. So those are brain computer interfaces. And then the other side of this are things that, uh, modify how our bodies work to enhance them beyond what evolution was able to discover. And so this would be, uh, for example, one of the companies that we backed is working on, uh, a gene therapy that helps, uh, people, uh, sleep less. And so there's a small percentage of the population have what's called a short sleeper mutation, which lets them sleep for like a couple hours a night.

Speaker A: I am so envious of them.

Speaker C: I know everyone is.

Speaker A: You give me a pill to help me join them. I would, I would make the Faustian bargain. So I'm with you.

Speaker C: It's a great, it's a great, it would be a great product. So, you know, if you could, if you could become like. And the thing is, they're well rested and they're actually healthier than the average person. So it's a very beneficial mutation that just didn't happen to spread. And so there are lots of mutations like this. And there's no reason that we can't intentionally move ourselves, uh, towards, uh, a more powerful version of what we got sort of, ah, dealt genetically, randomly. Um, and so those are the two paths of becoming more than we are now, so that we can stay ahead of what AI is becoming, so that we stay at the center of our own civilization.

Speaker B: And just to make sure I understand

Speaker A: this correctly and to make sure it's framed properly, the bci, the brain computer interface component of this, is what much of the TED community was buzzing about afterwards. And the idea not, uh, to be

Speaker C: reductive, that was all that was in the TED talk. So the TED talk I had, you know, there's a lot to say about all these things. So I had to really narrow it down to just talking about brain computer interfaces and how that represents a direction for us to expand our consciousness into and to become more than we are.

Speaker A: Is it fair to characterize it though, in broad terms, the idea is to effectively join, like, have a human join with AI so that our, our brain has some kind of literal mind meld with some type of AI system, so that when we are thinking, we are enacting human Agency and we're doing stuff, we are able to effectively, like, dial up an LLM to get smarter or to, to, to sum it up, facts.

Speaker C: Well, we, I mean, we dial up the LLM today to get smarter. I think this is just like, this is helping with the lag. The thing that makes AI feel separate from us right now is the fact that you like, have to type on a computer and then you have to wait for it to, you know, send you the, the letters back and then you read the letters with your eyeballs. And if you could just think and like, the answers come, then that lag disappears and eventually it just gets really hard to tell where your thoughts end and where your access to the vast repository of all human knowledge begins. And so that would be an instance of how this we've made manifest. And I think, uh, there's many different futures that involve this enhancement. Even if you just narrow it to the brain computer interface part, there's what you described, which is we reduce the latency between us and AI to the point where it feels seamless. And then there's other, I mean, weirder and more interesting versions where like our, our, our neocortex is built on top of, you know, the brain stem. And so we've been adding layers of complexity. And the, and the neocortex didn't like, erase the brain stem. It was just literally stacked on top of it. And you can imagine another layer that goes outwards made of silicon that expands and our consciousness expands upwards as it has, you know, from the, the stem on upwards, growing, you know, our, uh, who we are all the way out. And so you could imagine a bigger us is the, is the outcome of, of this sort of enhancement.

Speaker B: So, so, uh, we had.

Speaker A: Thank you for establishing kind of baseline of the, the, the framework here. Um, going back to those 1, 2, 3 tenants, the first one, I think that I easily agree with you. The idea that we can't stop AI development, right? We can't put the genie in a bottle. It's hard to imagine any scenario where AI development just halts. Even if there was some moratorium by governments across the globe, which is wildly unlikely, all it takes is one country or one actor to keep going, and it happens, right? So I think that's not realistic to imagine we could stop development. Can you share more on why you think the second point on, um, why it's like we can't control it? Um, why are conventional guardrails, um, regulation? Why in your mind is that laughably insufficient to, um, tame the beast I

Speaker C: mean, I hope somebody figures it out. But I think the biggest challenge that I see with all of our efforts to control AI so far is they don't work. Um, and we don't understand them, we don't know why they don't work. And so Anthropic had a really exciting announcement recently that they, they managed to decrease the rate at which Claude will choose to blackmail users. And it's just like such a crazy world to live in. Where was there a parade?

Speaker B: Was there a parade?

Speaker A: I missed it.

Speaker C: Yeah, I missed it too. It's a Good news. The AIs are blackmailing us less now, but we don't really know why. And like, and it's, it's very unintuitive. And so these, these are just. You have to think of these as like alien minds.

Speaker A: Yeah.

Speaker C: That look like us on the surface, but underneath there, anything but. And we don't know what they want or how to control them. We have no grounded theory that explains, you know, what it means for a thing to be aligned. And then we have no way to manifest that theory into an implementation that guarantees the system will stay aligned. And as these systems get more powerful, it gets more concerning because, like, you might think, oh, uh, well, if they're 99% aligned, we're probably fine. But like, it's kind of like being like, well, if 99% of your body doesn't have cancer, you're probably fine.

Speaker A: Yeah.

Speaker C: You know what I mean? Like the, all of the, all of the gaps in between what we want and what it chooses to do become really big problems, the more leverage you give a system like this.

Speaker A: I'm going to do my best to summarize and synthesize a lot of the common reaction and response I heard from the talk and your session and would love you to respond and push back to that reaction. And broadly speaking, what I heard is folks saying, okay, I get AI will keep getting more powerful. I get. We are don't have any great ideas of how to control it, but dear Lord, this seems like a really extreme solution.

Speaker B: Right?

Speaker A: Is it that you can't beat them, join them? The idea of surrendering, the idea of surrendering our brains is that the, the idea of surrendering our brains to the AI feels like a massively, um, uh, fraught with risk solution that folks are blanching at. Ah, uh, what's your response and reaction to that? I think, fair to say common reaction.

Speaker C: Jeff, people are surrendering their brains to AI right now, and I'm the resistance. Like, we have to stop surrendering our brains to AI. And I think the only way we manage to stop surrendering our brains to AI is if they stop out competing us. And I think the only way they stop out competing us, since we can't, you know, we can't ban all AI or we can't stop it from getting smarter, we can't stop the chips from getting faster, is we have to get faster too. And like, fundamentally we have a better algorithm inside our heads. Like our brains are a million fold better than AI is per unit of training data, per watt of electricity than any AI system today. But we're running on hardware that's 300 million years old. And so to the extent that we can upgrade our hardware, and I think the extent is probably very large given how fast we're gaining knowledge on brain computer interfaces and how fast we're making progress on, uh, know, uh, biological, on many different biological fronts, I, uh, think that future is one that makes it much more attainable for us not to turn over our brains to the AIs. And so like, I'm here for the humans, if that makes sense.

Speaker A: One question that folks have is, okay, let's grant the premise. Let's grant this is the inevitable conclusion in like a, the, the narrow path, if you will, for humans winning. Um, what, like who's building this? And a lot of concerns folks have is, look, we've seen in the past big tech be careless with our data. And so in this case, we're arguably surrendering literally our most precious data possible, like our minds, our consciousness. Surrendering that or not surrendering, we're were potentially, um, I don't know what the right term is, where that's being ingested, that's somehow being used by, used to kind of create this kind of um, amalgam of human and AI. How do we trust the companies building this will be, um, responsible stewards of that brain data?

Speaker C: I think it needs to be the most regulated industry in the world in the long term. I mean, right now we're in the early days and so regulation would kill it before we had a chance to even fight back against the AI. Um, but I think as these companies succeed in building, um, a hat we can wear that helps us use computers with our thoughts and helps us enhance our ability to think. Um, those kinds of devices will need to be regulated. And I think eventually actually those kinds of devices and the whole infrastructure, uh, behind them will have to become a public utility, like the water system or electricity or something where it's owned for the common Good by all. And it's subject to extremely stringent and public rules about how everything you know is used and how you ensure your, your thoughts stay private. And, and it's, uh, it's not something that big tech controls. It's something that belongs to all of

Speaker A: us is the idea.

Speaker C: So the companies are working on this. Okay, so. And that's the other half pioneers. So I do think the pioneers, like, for this are taking a lot of risk and they should get a reward. Like, they should definitely be able to build a big company and make a bunch of money. And then just like we do with the railroads and just like we did with the electrical system, we say, thanks, guys, now it's time for this to become a public good. And so just, you know. Yeah, go ahead.

Speaker A: Sorry, go ahead.

Speaker C: Uh, I was gonna say I think the, the, the company is, the company I think that is at the forefront of all of this is, is Science, which is a company by Max Hodak. Max was the co founder of Neuralink, uh, and he was the president of, for a while and he, uh, helped to build a lot of the technology. And he realized there's a fundamentally better way to build a brain computer interface than putting a bunch of wires in your brain. The problem with wires in your brain is that every time you put a wire in, uh, it kills neurons. And so if you want to have a thousand wires in your brain, you're going to kill 10,000 neurons. And the ratio is really bad because you're killing more neurons than you're reading from. And so call that the murder coefficient. Basically, you put so many wires in your brain, there's no brain left. So this is not a, uh, it's not a real path to getting to something that works for our biology. And I think the other issue is that anything foreign you put into the body, the body treats like a splinter and like, tries to push it out and it puts scar tissue around and everything. And so what Max figured out is that you can have, uh, an interface to the brain that doesn't hurt the brain at all. It doesn't touch the brain. It just takes little genetically engineered neurons and puts them on a thin film right on top of the brain. So nothing goes in your brain. And those little neurons do what neurons do, they grow dendrites that, that connect into your deep network of neurons so that they can talk to the electronics in the film. And this way there's no implant going into you. There's just basically like a sticker that sticks in your brain. And the sticker is covered in real neurons that grow and connect with your neural network. And that enables you to think and not have a disruption in your thoughts. And it's a much more safe way of getting a neural implant. Now again, that's one way of doing it. That's the most aggressive way of doing it. You can also have, there's other companies working on ultrasound, sort of like hats that you wear that can read and write to the brain. So there's a lot of different ways that I think people will delighted, Scott,

Speaker A: that you did not describe a Borg. Like, uh, I was, I was, I'll admit I was envisioning the Borg here. So I'm somewhat reassured that's not the end game you have in mind.

Speaker C: Uh, Jeff, the point is to stay human. The goal is to stay human and keep humans at the center. And like, honestly, we're losing that. That's part of why I want to talk about this because paradoxically, the way for us to retain our humanity and to retain humans at the center of our civilization is that we have to become more than we were in caves.

Speaker A: Right?

Speaker C: And that's a hard thing to accept, but it really does beat the alternative. And I think there's a way to do it that's not invasive and that is like just, uh, you know, there's already a lot of stuff. You already wear a hat and uh, you already take, you know, probably some vitamins every day and stuff. So there's a ways to do it that are like things that we're accustomed to, um, and also provide the benefit of keeping humans at the center.

Speaker A: How are you thinking about who is going to do this? I'm assuming it's a volunteer. I'm guessing you're not saying all of humanity is going to be forced to do this, right? It's a volunteer based thing, I'm guessing. Um, and how are you thinking about inequality in this? And will. Will it be will. Will it. If potentially, if there's already a haves and haves nots, will this kind of create a joins or merge and merge knots? And if people are becoming, if those who have means to do this, if they're becoming literally superhuman, will that exacerbate current inequalities and structural issues? How are you thinking about that challenge?

Speaker C: I think this will be ultimately a force for reducing inequality because so I have a friend who does a lot of work with the Gates foundation and does, uh, a lot of work in Africa and he actually had a meeting with Bill Gates and, and then he had to in Seattle. And then he had to fly to Rwanda. And he was in rural Rwanda the next day. It was like, you know, late at night. And he got into this very remote village, uh, and was in this, uh, dirt floor hut, basically. And he noticed that the little girl next to him, M. Was doing her homework on the same iPhone that Bill Gates was using. And so the thing about the technology, when you scale it like this, is that you have the same phone that Elon Musk has. Like, you have the same. You use the same search engine that Jeff Bezos uses. It's all the same. It's like the best force for. For, uh, equality and equity in the world. Because by making, you know, a little piece of film, you can make it really cheap. You can make a lot of them. And it's something that everyone can have access to if they want it. And I think the. If they want it is also a really important aspect of this, is that, you know, people are going to want different levels of integration. And that's okay. People want that Today, like, we have the Amish who prefer to live a life that's, like, really much closer to our roots as an agrarian society. And then you have people who are, like, super into tech and have all the gizmos and gadgets, and that's okay for them too. And so I think in the same way, this technology will be a spectrum of people who prefer a vanilla experience. People who like to have a hat that, like, they, you know, enhances their ability to think. People who like to take special vitamins that, you know, give them an IQ of 400. And then there's gonna be people who wanna go all the way. Uh, and I think all of those are okay, and they will create differences between people. But we already live in a society full of lots of differences between people that kind of mirror that. And it's been okay so far. And I think as long as we can appreciate what each other has to offer, it doesn't end up changing much about the society we have today.

Speaker A: You're a accomplished investor. Big part of investments is thinking about the upside, downsides, rewards and risks. What do you see as the biggest risk in all of this? What's kind of the downside scenario?

Speaker C: I think the biggest risk is if it doesn't happen. Because if you just, like, imagine that everything goes great with AI. Like, imagine if everything goes. Like, there's a lot of. I mean, we can talk about the things that go poorly with AI first, and then we can talk about what if it goes great. So I'D say the things that go poorly with AI, like we're functionally equipping everyone in the world with, you know, uh, a data center full of geniuses to do their work for them. And I think that's fine if the people have good intentions, but it also amplifies the abilities of the people who have bad intentions.

Speaker A: Yep.

Speaker C: And in a world where, you know, they, they talk about a polite society as an armed society, I think that might be true. But if everyone's actually armed with a nuclear bomb or something, then it's not actually a polite society anymore, it's just a terrifying society. And that's kind of what AI at its, at the limit anyway starts to do. It starts to equip people with the ability to do much more with their intentions, whatever those intentions are. And it amplifies the very small number of people who have deeply antisocial intentions. And so I think there's many tail risks that we're all going to face as a society of people who just want the world to burn and they're equipped with a lot more power to do that because of how technology is an amplifier of intention. That's like one, one class of risk is that we, you know, we're navigating society. But, uh, like, let's say all of that goes perfectly well. You know, we find a way to make sure that no one does anything bad with A.I. that's, that's catastrophic. And then we get to the second phase, which is like the AI itself is a thing that we don't really understand very well and we can't really control super well. And it behaves in all kinds of concerning ways with respect to its desire to stay, you know, in existence. And so there's many scenarios you can imagine where AI is given a good goal, but it accomplishes in a bad way or prioritizes its own well being above humans. And so that would be like another thing that could go super wrong. Like it just really wants the power for the data center, so it happens to turn it off for the hospital and then. Sorry, go ahead.

Speaker B: Oh no.

Speaker A: I know there are tons of risks

Speaker B: with AI so I could see that.

Speaker C: Okay, so that's the AI risk bucket. But let's just say all that goes great. Okay, the AI risks, that's all fine. It's just the best ever. AI is just here. It loves us, it's here to help us. And in that scenario, gradually we are giving away more and more of our institutions to the control of A.I. you know, we're imbuing because it makes sense. Like we have an A.I. that we know is going to do the right thing and it's much more impartial than our politicians are today. And it's much more competent than they are too. And it wouldn't be better if they just made more of the decisions that we rely on for our day to day life. Wouldn't be better if it ran the water system so that there were fewer mistakes and the water was cleaner and so on and so forth. And eventually we just, we kind of hand our society over to being run by minds that are much brighter than ours and we end up in the passenger seat of our own civilization. And I think that human sort of gradual disempowerment of humanity I think is a really dark future for us. And that's why I would say like the biggest risks for me are the risks of not enhancing ourselves, not, not, not creating the circumstances where humans can stay in the center of humanity. And now, okay, that's like the disclaimer. And now you want to know what are the risks specifically of like brain computer interfaces or you know, and I would say there's lots of there, there's lots to think about there too. So I'll, I'll put it to the side. The risks of the ones that are like a vitamin you take because that's normal medicine. We have the FDA that'll go through clinical trials. Of course I would say the, the risks of brain computer interfaces that are hardware devices like ones that you wear are, you know, initially going to be like what are the side effects? Can you, if you wear it for 12 hours a day, does it, you know, give you a big headache or something? So those are things that'll have to be tested. The medium term risks get into data, which is what you started to hint at earlier, which is if you have a device that can read your thoughts, how do you make sure that your thoughts stay private?

Speaker A: Yes.

Speaker C: And you know, I think that's where we really need strong regulations to ensure that private thoughts are private and that there's a very, very, uh, rigorous approval process in place for anything that's going to read your mind. Um, and I think at the limit, you know, to make sure it goes really well, I would say we can't have some giant mega corporation that everyone pays subscription to in order to access their own thoughts or something. That's like a dystopia.

Speaker A: Completely agree with, um, Scott, I know we're running short on time. Last couple of questions for you. What? Um, uh, I actually Asked this question in Vancouver at the, at the off record session. I asked, uh, let's imagine for a moment this happens. Um, what is the wildest implication that is also plausible? So I'm not saying super long tail, 1% possibility, but what do you think? Okay, let's look. If something is plausible, what will kind of melt a lot of minds of listeners of uh, this is likely to happen in the future?

Speaker C: Well, uh, so if conscious is a physical process, which it has to be if we're going to be interfacing it with technology, that means that you can port it to chips and it means you can connect it together and so you can have a shared experience with your closest friends or with your loved ones in a way that's much deeper than language allows. And you could even extend that a little further out if you're willing to, which is you have a shared experience with non human animals like your pets or like dolphins or something. So you could understand truly the consciousness of all different animals on earth in some way. That's impossible for us to do today. That's like, I think a pretty wild aspect of that future. Maybe the other one is, um, if your conscious can be moved off of your brain or outward, then like you can have a backup, you know what I mean? If something bad happens, I was going there.

Speaker A: I was wondering if you're hoping you're

Speaker C: going there, your consciousness remains and I think that unlocks space travel because you're never going to send this, you know, juicy Earth, you know, ball of flesh all the way across the stars to make it across the stars. We have to solve this problem anyway. And so I think that's the other thing that would be really interesting about this is to be able to store your mind and wake up on a planet far away is the other thing this enables.

Speaker A: And how, Scott, do you think this changes what it means to be human? If it, if it does?

Speaker C: I think what it means to be human has changed a lot since we were, you know, hunting stuff and, and living in caves and you know, barely had anything at all. I think they would look at us today and not recognize themselves as human. So I, I think that will be true of whatever the future looks like. I think we'll still see the, the core flame of humanity in each other and ourselves as we become more enhanced than we are now. Just like I think those primitive ancestors recognize us and be like, yeah, it's human, but wow, what's all that extra stuff they're doing? And so I think we'll probably have that same experience. And I also think it'll, it'll look like magic in the same way that tribal societies would view our lives today and be like, wow, there is just magic everywhere.

Speaker B: Scott, I so appreciate you taking the

Speaker A: time to unpack all this. I find it enthralling. This is the stuff that gets me excited about AI because the are the stakes that we're playing with in the future. And I think whether, um, people agree with this possibility or not, I appreciate you engaging in this conversation. I find it that fascinating. Thank, uh, you so much for taking the time. Really enjoyed it. Jeff.

Speaker C: This was really fun. Great to spend time with you today.

Speaker A: Well, there you have it.

Speaker B: Thanks again to D. Scott Phoenix. Thank you to our show's producer and editor, Jasper Chua. Thank you to Garrett Lang for our theme music. Uh, if you are a brain AI merger, uh, thank you for listening. I hope your AI half of yourself enjoyed it as well. Please send it to a friend. Radio 5 stars. Thanks again and see you next time.