Four Fs Podcast · 2026-06-05 · 1h 44m
Key moments - from our scoring
Substance score
54 / 100
Five dimensions, 20 points each
Dr. Craig Kaplan brings extraordinary credibility to AI discourse: he studied directly under Herbert Simon (who co-created the first AI program in 1956 and won the Nobel Prize), and has maintained continuous involvement in the field since 1985 - giving him a rare vantage point on how foundational ideas from the 1950s-80s are now driving modern large language models like GPT and Gemini. In this episode, Kaplan walks through his journey from UC Santa Cruz (double major in psychology and computer science) through Carnegie Mellon's cognitive psychology PhD, where he witnessed the birth of neural networks alongside symbolic AI approaches. He clarifies the distinction between traditional AI (rule-based systems) and machine learning (pattern discovery through data), explains why neural networks failed to scale in the 1980s despite correct core theory - the bottleneck was computing power, not algorithmic soundness - and argues that Moore's Law eventually unlocked dormant ideas. Kaplan estimates an 80% chance AI becomes humanity's best outcome, but sees the 20% downside risk (existential) as unacceptably high. Critical for founders, CTOs, and strategy leaders evaluating AI's trajectory and safety implications, particularly those building on LLM foundations or making long-term bets on reasoning-enabled systems.
AI is a broad field encompassing rule-based systems you program explicitly, reasoning, planning, and perception. Machine learning is a subset where instead of hand-coding rules, you provide a general learning algorithm and massive datasets; the system finds patterns and produces models (like LLMs) without humans explicitly knowing where learned information is encoded in its billions of parameters.
The algorithms were theoretically correct but lacked computational resources to run them effectively. Kaplan's office mate spent his entire PhD trying to get neural networks to recognize the letter 'A' - a task modern systems handle trivially. The breakthrough came when Moore's Law and increasing computing power (around 2010) made the old algorithms finally practical.
Concepts like reasoning systems that Simon pioneered were shelved due to computing constraints but are now being reintegrated into modern LLMs. As Kaplan notes, if an idea was fundamentally sound but lacked computational power, it eventually resurfaces when technology catches up - making these decades-old frameworks surprisingly relevant to GPT, Gemini, and newer reasoning-enabled models.
He estimates an 80% chance AI is the best thing ever to happen to humanity, but emphasizes the 20% downside risk of existential danger is unacceptably high - almost nothing else has a one-in-five chance of killing everyone, which is why he dedicates effort to improving those odds despite his optimism.
They had rooms full of IBM 286 PCs and could only move a few inches per hour, stopping frequently to process what they saw. The same algorithms today, running on modern computers, could drive at 180 miles per hour - illustrating how computing power, not algorithm design, was the limiting factor.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode has pockets of genuine substance - the democratic AI architecture proposal, synthetic data as a path to superintelligence, and the agent-to-community progression - but the first quarter is almost entirely biographical small talk, sibling chat, and college anecdotes that yield zero operational value. When insights do arrive, they are diluted by basic explainer content (Venn diagrams of AI vs ML) that any informed listener already knows.
what if you designed the most powerful AIs, not as individual black boxes that are hard to predict, but rather as communities that work together like a democracy, and they're made up of millions of less intelligent AIs and maybe some humans as well, working together
the AI that had the highest score on this was, uh, developed by Xai. It was one of Elon Musk's versions called Gro Heavy. And what Gro Heavy did behind the scenes, it had a community of agents
The democratic-AI / collective-intelligence-as-safety-mechanism framing is Kaplan's most distinctive contribution and genuinely uncommon in mainstream AI discourse; the argument that values outrank regulation as the key safety lever is also underrepresented. However, large portions recycle standard talking points - AlphaGo playing itself, Moore's Law enabling ML, Asimov's three laws, the child-parent analogy - that circulate widely.
If we don't model good values, it'll be really good at absorbing bad values, and that will not look good for us
I think in the future that personal data is going to become extremely valuable, like more valuable than diamonds or anything
Kaplan is a legitimate deep practitioner: 1985 CMU graduate researcher under Herbert Simon, founder of multiple collective-intelligence companies including one that ran a hedge fund off crowd wisdom for 14 years with Schwab, TD Ameritrade, and Nasdaq as partners. He has genuinely done the thing at scale. He is not a career podcast guest, but his current role is more evangelism and Substack writing than active building, which tempers the score.
I've occupied a tiny little niche... collective intelligence systems... I actually built these, uh, systems. I, you know, founded a couple companies that focused on these. And we. One of them I ran for 14 years on Wall street
We powered a hedge fund off of millions, the intelligence of millions of everyday investors that you might find on Schwab or TD Ameritrade or nasdaq. We were partners with all those firms
The episode contains genuinely good concrete anchors - the 286 IBM PC self-driving van doing inches per hour, the IBM $1-vs-$10,000 quality-control ratio, the Grok Heavy Humanity's Last Exam detail, the 1956 conference with 11 scientists - but the Wall Street hedge fund claim, which is the most impressive credential, is never backed by performance figures, fund size, or verifiable metrics, leaving the biggest claim frustratingly vague.
IBM at the time was the largest software developer in the world, bigger than Microsoft or anybody. And they did all kinds of studies that showed if we just spent $1 extra more in the design phase, we could prevent an error that cost $10,000 on average if we caught it when the product shipped
We had self driving cars at Carnegie Mellon in the 1980s... there'd be this van... filled from the floor to the ceiling with, you know, 286 IBM PC, you know, computers... the whole van only went a couple inches an hour
The host is genuinely curious and occasionally sets up productive moments - the 'tell me why I'm wrong' invitation and the quantum follow-up show real intellectual engagement - but he spends nearly a quarter of the runtime on biographical small talk, makes factual errors (Santa Barbara vs Santa Cruz), and consistently validates rather than challenges, letting substantial claims like the Wall Street hedge fund performance go entirely unprobed.
Was it ever a, you ever get find yourself frustrated when you kind of, you know, like I said, you've been kind of working on this idea or had this theory for a lot of years
Yeah, for sure. I mean, if you're not living off the bank of mom and dad at that point, like, that's, uh, that's. That opens up all kinds of possibilities
Computed from the transcript - who did the talking, and the words that came up most.
Superintelligence isn’t science fiction anymore. In this conversation, world‑leading safe AI expert Dr. Craig A. Kaplan explains why AI is already behaving less like a tool and more like a worker - and why the next jump to true superintelligence could change everything about how humans live, work, and make decisions.We talk about how AI went from clunky 1980s experiments to today’s large language models, why “machine learning” is only one part of a much bigger AI story, and what it really means to treat AI as an intelligent entity rather than a gadget. Dr.
Transcribed and scored by The B2B Podcast Index.
Jeff Cluff: Doctor Craig Kaplan didn't just study AI, he sat in the room where it was being invented. As a graduate researcher at Carnegie Mellon, he co authored work directly with Herbert A. Simon, the Nobel Prize winning economist who co created the very first AI program in 1956 and coined the term artificial intelligence. That heritage, combined with Craig's decades long obsession with collective intelligence, gives him a perspective on AI that almost nobody alive can match.
Dr. Craig A. Kaplan: My parents always thought like education was the most important thing. Might be wearing, you know, jeans from Goodwill or something, but I may have been the poorest dressed kid at this elite prep, uh, school. If you could get into the school, you should go. I ended up at Carnegie Mellon and I went there because of Herbert Simon, who is a Nobel Prize winner and one of the pioneers who invented the field of AI. And I ended up working with him. I just had a ball. If you build a super powerful intelligence that's, you know, a thousand or a million times smarter than the smartest human, all those other problems kind of get solved. I think we have company now. We've created entities that are almost as smart as us and almost certainly will become smarter than us in the future. The majority chance, like 80% chance that AI is the best thing ever to happen to humans. But so the only reason I talk about the dangers is because 20% is way too high a probability. I mean almost nothing has a 20% chance of killing everybody. That's one in five.
Jeff Cluff: Dr. Kaplan thinks there's an 80% chance AI is the best thing that ever happened to humans. But that other 20%, he has a plan to change those odds. So you're in California, but you haven't, you're not always from California, from what I can tell or way or, or where are you from? California. I should just ask where are you from?
Dr. Craig A. Kaplan: Yeah, I was born in California, in Berkeley, and uh, lived the first 14 years there. But I've spent time in New Hampshire and Pittsburgh, went to grad school at Carnegie Mellon. So that was Pittsburgh. And you know, I've traveled around. I keep coming back to California. It's beautiful. And a lot going on with technology. So that combo is hard to resist.
Jeff Cluff: Yeah, for sure. So do you do your undergrad in Pittsburgh as well?
Dr. Craig A. Kaplan: I did my undergrad at UC Santa Cruz, uh, which is pretty close to where I live now. And um, yeah, I just ended uh, up double majoring in psychology and computer science, which is kind of a weird
Jeff Cluff: combo, you know, I mean, looking forward, it, it does, right? If you're looking okay, why would you major in psychology, computer science. But looking back, um, and, you know, it adds up. Makes sense. I could see how it fits right in here now.
Dr. Craig A. Kaplan: Right. It was kind of perfect for what I ended up doing. I guess, sort of my interest took me there.
Jeff Cluff: Yeah. Were you, were you pretty heavy into academics? Were you or were you have a nice, Nice, uh, balance to enjoy some of the, the other parts of college life?
Dr. Craig A. Kaplan: Yeah, yeah, no, it was great. Um, I had gone to a very kind, um, of difficult, um, program prep school for high school, uh, in New Hampshire. And so coming to UC system was. It was like a cakewalk actually. My senior year of high school was so much harder than my first year of college. So I just had lots of time to have fun and, you know, make friends and, you know, hang out in the dorm and have philosophical conversations, all that kind of stuff.
Jeff Cluff: That's cool. Um, that, I mean. Yeah, that really is saying something. I mean, it's usually a heavy learning curve, you know, even for the brightest folks to, to uh, jump into any sort of college. Was this like, like a boarding type school that you're at in New Hampshire?
Dr. Craig A. Kaplan: Yeah, I went to Phillips Exeter Academy. It's the same place that Mark Zuckerberg went to, actually. And, um, it's funny, you know, he started. This is all speculation on my part, but he started, you know, the company Facebook. And I gotta believe that he got the name from a little book that you had at Exeter, as in high school, where they had a picture of all the kids in the school and it was called the Facebook. And you know, kids would spend like hours on their bed sort of looking through, trying to find out where the pretty girls were if you were a guy or vice versa maybe. And um, it was like this bible that you sort of, if you wanted to know who was who at the school, you went to the Facebook. And um, because he went to the same school, and I'm sure it's the first thing everybody got at the bookstore was bought a copy of that. I just wonder if if that sort of was the source of the name of his company. I, I don't know if that's true or not, but it makes sense to me.
Jeff Cluff: I mean, it, it's hard to, I mean, you know, and even if it's not consciously firm him, like, it's, it's hard to remove, you know, to, to separate completely from those kind of those core influences. Right. So that's funny. There has to be some sort of connection there. The Facebook. Yeah, I like it. Um, was that. So, so how did did you, uh. Was this something that you sought out, this education at Phillips Exeter, or was this something, you know, like, um, your parents encourage you to do it? You can't get much further away, staying in the country than going from California to New Hampshire.
Dr. Craig A. Kaplan: Yeah, I think I was pretty lucky in the sense that my parents always thought, like, education was the most important thing. Right. So we would scrimp and save, and, you know, we might be wearing, you know, jeans from Goodwill or something, which we did, going to school. But, uh, I may have been the. The poorest dressed kid at this elite, uh, prep school, but they were gonna, you know, if you could get into the school, you should go and, you know, find a scholarship or whatever it is that you could do. Um, so they were big believers in education, and that sort of carried over. So I'm sure it was their idea initially. And then I kind of got hooked on it. And, um, you know, I love learning, so it was a great place to learn. Really smart teachers and really smart students. I mean, yeah, it was just a lot of fun. And so I had a great time there. I had a great time in grad school. It was the same thing where you had these super smart professors that were, you know, founding the field of AI and, you know, doing this stuff that was, you know, decades ahead of what anyone was doing. And it was just like kind of brain candy to be around those guys.
Jeff Cluff: Talk me through, if you can, kind of the process, uh, or the timeline from when you went from college. Uh, UC Santa Barbara. That's right.
Dr. Craig A. Kaplan: Is that what it was? UC Santa Cruz. Yeah.
Jeff Cluff: Sorry. Okay. UC Santa Cruz. So talk me through the time when you went from. To. From college at UCSC to, uh, to going to grad school. Is it a direct transition or did you work somewhere between?
Dr. Craig A. Kaplan: Uh, no. I mean, I went to ucsc, um, and was interested in psychology, interested in computer science. Um, I took a class in psychobiology that talked about how the brain worked. Um, um, and that's when you learned about neurons and dendrites and how these all, you know, hooked up in their inhibitory, you know, responses and excitatory responses. And the whole thing just seemed really fascinating to me. But I sort of figured that, you know, I would graduate college and get a job being a programmer, actually. So computer science was like my backup plan of, you know, you can make a living. Right. That was very important, sort of how I grew up, that you wanted to be able to, you know, support yourself. Um, and my last year at, uh, UC Santa Cruz, I had a professor Uh, I did really well in his class because I was just really interested in the subject matter, which was cognitive psychology. And that's like human attention, memory learning, problem solving. It's kind of basically how the human brain works. And, uh, so this professor took me aside sort of at the end of the year, or the end of his class, I guess it was first semester, and said, so, what are your plans? You know, you seem really good at this. I said, well, I'm gonna go get a job. You know, I need to make money. And he said, well, why don't you go to grad school? And I said, I can't afford grad school. I could barely, you know, work my way through college. He said, no, no, you don't understand. They'll pay you to go. And that was a, that was a light bulb for me. I hadn't heard of that before. Uh, but it turned out that a lot of these programs, if you got in, they gave you a full ride and they gave you a stipend. So that changed everything. I was like, wow, I've been working my way through school and paying you, and now they're going to pay me to go to school? Yeah, I'll do that. So that's kind of how I ended up at Carnegie Mellon. And I went there because of Herbert Simon, who is a Nobel Prize winner and one of the pioneers who invented the field of AI. So in 1956, the field was named, and there were 11 scientists and two of them came from Carnegie Mellon. Uh, Herb was one of them, and I ended up working with him. And I just had a ball. It was, was great. And, and they paid my school, which was, you know, for, for a young, you know, 20 year old or whatever, that was amazing.
Jeff Cluff: Yeah, for sure. I mean, if you're not living off the bank of mom and dad at that point, like, that's, uh, that's. That opens up all kinds of possibilities, which, ah, you weren't. You said you were working your way through school. What, what kind of jobs were you doing at the time? Through college?
Dr. Craig A. Kaplan: I, uh, mean, I had summer, uh, jobs. So I worked at a hotel. Uh, I like this because, um, you know, I listened to Jensen Wang, the CEO of Nvidia, and he has some great stories about working his way up from a busboy at Denny's and cleaning toilets. And I can totally relate. I had the same job at this hotel, uh, where I had to clean the bathrooms and I was the lowest person on the housekeeping staff. I carried the laundry for the maids. So the Maids would make the beds and everything, but I would go from room to room with these giant garbage bags and, you know, just. You load up as much as you could carry and you'd like, carry it down to housekeeping where they did the laundry and anything the maids need it, you, you know, help them out. I was like the maid's assistant, right? It was called a houseman was the job. So I did that. I did bathroom stuff, I did busboys at weddings. I, you know, served the fancy steaks. Two weddings, you know, every Saturday, every Sunday. So I really got to see what a wedding was like from the opposite side, not getting married part, but like serving the people and all that. So, I mean, I learned a lot. Um, but I just did whatever I needed to do to sort of make money basically.
Jeff Cluff: So, um, mentioned, you know, how you brought up a couple of times and I'm guessing this is, uh, these things are related, but, uh, what kind of work do you folks do?
Dr. Craig A. Kaplan: So my dad was in real estate, um, and it was kind of an up and down thing. Like it was boom or bust that when we were really little he was doing fantastically and it's like, well, he had it made and then he'd get into some deal and then we'd be almost on the verge of bankruptcy and you know, struggling to pay the power bill kind of thing. Uh, so it was one of those kind of, um, you know, upbringings where you had a lot of volatility. But my parents were, you know, had a strong bond for each other and ah, there were five of us boys and my mom was an awesome mother and she would do whatever. I mean, she, she raised us and remodeled apartments and painted fences and knocked down walls and did whatever it took to sort of, you know, put supplement whatever my dad was doing. So it was an interesting upbringing. Uh, uh, and it was a kind of, you know, rough and tumble. With four boys, you're always sort of getting into things with your brothers and you know, I, I don't know if you have, uh, siblings, but you may be able to relate 100%.
Jeff Cluff: I have five brothers myself.
Dr. Craig A. Kaplan: Okay.
Jeff Cluff: I can, I can definitely relate. I mean, lots of stuff just gets damaged and broken and we were like reasonably good kids, you know, but like, you just like five boys, like, it gets broken like, like this just. It's just boys are different. You know, as a parent now, I can definitely see the difference, um, in how I acted and you know, how my kids.
Dr. Craig A. Kaplan: It's just.
Jeff Cluff: Yeah, that's, that's fun.
Dr. Craig A. Kaplan: Do you have um, daughters and sons now or.
Jeff Cluff: No, sorry, I just have two daughters.
Dr. Craig A. Kaplan: Two daughters. Okay.
Jeff Cluff: Yeah, the. It's marked Markedly difference.
Dr. Craig A. Kaplan: Yeah, markedly different.
Jeff Cluff: Like, it's like.
Dr. Craig A. Kaplan: Yeah, I can relate. Exact same thing. Uh, I had only boys growing up, right. Four brothers. Um, and then I have one daughter and two sons. And my daughter is different. It's like I somehow missed the course on, you know, sisters and relating to daughters. So I'm figuring that all out.
Jeff Cluff: Um, I'm sure, I'm sure you're. And that's the beautiful. That's the beautiful thing of it, right? It's like, ah. I mean, I'm not that I have it figured out at all. 0%, uh, of it is figured out. But, um, but it's, It's. That's the. It's a new opportunity to grow. Right. And learn and just not. I'm with you. I love learning. And so there's always, there's always something to learn with them. Um, yeah. That's cool. Um, okay, so this, this helps me paint a picture. Then I can see a little bit about, uh, about you kind of as a person. You know, you grew up, your folks are good people. They're working hard. You're. You're working hard, but you had a goal. Did. Did they. And it sounds like they instill in you, at least in some ways, this. The importance of. Of learning. Right. Because they, they decided it was important enough to send you to New Hampshire and support you in that. Um, okay, so how did you choose. You said you. How did you choose Carnegie Mellon, uh, over any other schools? Or were they the only option?
Dr. Craig A. Kaplan: No, I mean, it was amazing. So, um, applying to college actually was harder than, for me, than getting into grad school. Part of it was I did really well at UC Santa Cruz because it was, you know, it wasn't at the same level probably as my high school, so it was easier for me. Um, and so then you do really well, and then you look good to the grad schools. But no, I got into Stanford. I got into Carnegie Mellon and got into pretty much all the top schools, and they all were offering me money, so I chose Carnegie Mellon. Really? Because of Herb Simon. I had seen an article. AI was a brand new thing. Uh, it was not mainstream like it is now. It was like a tiny little niche that only a few really geeky people knew about or cared about. And I just thought this was amazing. Like, I thought it was really cool to understand how humans thought. But here it was actually building a machine intelligence. Um, and it's Kind of like, uh, I've heard Demis Asabas, who is the CEO of DeepMind, uh, when he's asked about how he got into AI, said, well, you know, I realized if you solve this one problem, you can solve all the problems, right? Because if you build a super powerful intelligence that's, you know, a thousand or a million times smarter than the smartest human, all those other problems kind of get solved or they get solved much faster. And I sort of realized that same thing. I was like, wow, this is never before in human history has this been possible where you could actually build an intelligence. I mean, this is crazy. So I sort of looked around for who were the top people, and Herb was right there. And so that's kind of what drew me because Pittsburgh, I thought. I was a little worried about going to Pittsburgh actually, because I visited and it was an overcast day and all the other graduate students were flinging open the windows and saying what a great day it is. And I thought, wow, this is not a great day in California if they think this is a great day. I wonder what it's like to live here, you know. But, um, but I got used to the weather and, uh, it was just a great experience.
Jeff Cluff: I like that. So did you already. So you knew you wanted to study AI, um, or about AI, but your specific, your specific studies weren't, were a little more specific than just AI, turns out. Is that right?
Dr. Craig A. Kaplan: Yeah. So I was accepted to the, um, psychology department. And there's a, ah, area of psychology, uh, cognitive psychology, which I alluded to earlier, which is all about how the brain works and the different functions that it does, memory and learning and problem solving and so forth. Um, now Carnegie Mellon was different than any other cognitive psychology department, at least that I knew of, in that the other ones would sort of emphasize more the psychology piece. And you do a lot of research on human psychology. But Carnegie Mellon, it was like they felt like you didn't understand the human psychology unless you could build a computer model of it. So almost every thesis, uh, had to have a computer model. I think the psychology department, the cognitive psychology department had more computing power per graduate student than most computer science departments. I mean, we each had our own. I mean it was well endowed, it had a lot of good grants because it had very famous people and they brought in the money from government grants and so forth. So each graduate student had like the latest workstation, private, on their desk. I mean, you just got it when you arrived. And, uh, you know, my office mate was doing early neural network stuff so, um, just Jeff Hinton, who people may know as sort of the godfather of AI these days, he invented um, or co invented neural networks along with some other people, which is kind of the algorithms that underpin all of GPT and Gemini and everything that people use. Right. So that work was just beginning in the 80s when I went to grad school. And um, some of the collaborators of Jeff hinton, like Jay McClellan, were there. So I got to be in his research group. So I got a really good sort of. It was a magical time for AI too. It was like the pioneer. You still had the old school approach, which was symbolic programming, the rules for AI, but you also had the new, brand new thing that people were skeptical about, which was this machine learning approach where the AI could just learn on its own. You just feed it a whole bunch of data and sort of the intelligence would emerge and there were these, you know, very heated debates that would go on, you know, in symposiums and people pounding the table and saying that doesn't work. And, and to be truthful, at the time, it did not really work. My office mate spent his entire, um, four or five years in grad school trying to get neural networks to recognize the letter A. That was like A, A M huge thing. If you could just recognize the letter A in whatever orientation it was, that was worth a PhD thesis back then. And nowadays, of course, those same algorithms can, you know, talk with you and program for you and do all that stuff. And really what happened was computing power increased. That was the thing that people didn't realize. In the 1980s, even though we had these powerful desktop computers, they were much less powerful than our phones and they just didn't have the oomph to kind of make the algorithms work. So all that had to happen was waiting and Moore's Law, doubling computing power every 18 months. And then at a certain point, probably around 2010, it sort of crossed a threshold where there was enough power. You could run those same old algorithms that you were doing in the 80s. And we had self driving cars at Carnegie Mellon in the 1980s. I used to go jogging and there'd be this van, all the seats and everything were ripped out. It had video cameras and it was filled from the floor to the ceiling with, you know, 286 IBM PC, you know, computers, which was the state of the art at the time. But the whole van only went a couple inches an hour. And then it would stop and it'd see a leaf and it'd be like, is it a leaf? Is it a person and have to think about it for like 30 minutes or whatever, and. And then it would inch forward those same algorithms. With today's computers, the thing could drive 180 miles an hour. So it really is a story of computing power increasing and then, um, enabling a whole new approach to AI, which is kind of the dominant one that we have today.
Jeff Cluff: That's fascinating. I had no idea that that was even happening, that people were even attempting that in the 80s. Um, I love it.
Dr. Craig A. Kaplan: A lot of the ideas that this is something that I think a lot of people don't realize. So one of the benefits I have is a long history. Right? So I started in, I, um, guess 1985 was when I was first at CMU and I built intelligent systems. And I've stayed in the field in one way or another, all the way till now. So that's a long period. And you can see these trajectories. And one of the, uh, key things is a lot of the ideas, just because they were invented, like in the 1960s or 50s or even Alan Turing in the 1940s, some of those ideas are absolutely the right ideas and incredibly relevant for AI today. It's just at the time people were coming up with them, they didn't have the computational resources and the computers available to really, you know, build the systems. But now what you're seeing is some of those ideas like reasoning systems. Herb Simon was big on reasoning. I mean, he was one of the guys that sort of invented that whole thing that's now finding its way back into AI systems. So it's kind of like if you wait long enough, if it was a good idea, but it just didn't have enough computing power, that idea comes back, and then you see it sort of, uh, you know, manifest in the latest generation of these large language models.
Jeff Cluff: I'd like to talk a little bit more about this, uh, the recent influx of reasoning into the AI models. But before we do, I was wondering if you could. You were describing something just now, and I really like the way that you said it kind of tangentially, uh, which is you were describing the difference between AI and machine learning. Um, and in other. I'm just wondering if you could separate those two for me.
Dr. Craig A. Kaplan: Sure. So AI artificial intelligence is kind of a very big field, and it encompasses systems that you program the rules into them and they can only do what you programmed in. Um, it encompasses reasoning, planning, perception, um, you know, all kinds of different AI systems. And then a subfield within AI is machine learning. And machine learning is the Idea that rather than having to program the rules in by hand for the AI, like write a computer program and you tell it exactly what it's going to do in every circumstance. Instead of doing that, you equip the AI or the system with a general purpose learning algorithm and then you just show it lots and lots of data and you let it run, you know, for weeks on end or whatever, looking for patterns. And at the end, uh, pops out a large language model, for example, that seems fairly intelligent. And people don't really know where the rules are, where the information is that it's learned. And that's one of the problems with these models is that they're very black boxy. You can't really see where the information that it learned is or where it's encoded. There's just billions of parameters which are just a numbers that are kind of analogous to in a human brain you have billions of neurons and these neurons are all interconnected and they have different strengths. You know, one neuron might be more tight, tightly associated with another. And, and you can see that in experiments where you ask people to free associate and you say bread and they say butter or you say salt and they say pepper. It's like, well, those two concepts are closely associated in the brain. There's a good strength there. So that same kind of idea multiplied by a trillion different, you know, connections is what you have in these large language models. But you don't know where the bread and where the butter is represented in that model. And it, so it kind of makes them difficult to predict, difficult to know what they're going to do, which is kind of a safety issue. But uh, from a efficiency standpoint, it's way easier to load in data and let them learn than it is to meticulously try to program in each piece of knowledge. And that was the big thing that happened. So machine learning is that subfield of AI, which because it was so successful sort of, especially after 2010, and it sort of really got a big boost on the algorithms in the 80s, but didn't take off until later because it was so successful. Now a lot of people sort of almost treat them synonymously as if all of AI is machine learning, but it's just. No, that was a really successful subfield of AI.
Jeff Cluff: That makes sense. So I want to just, uh, so I'm all the way clear. So if I was looking at this like a Venn diagram, the big circle is AI and there's a, there's a smaller circle within that circle. That would be machine learning. But there's no overlap. It's not like an intersecting, like, overlapping at all. There's no, there's no machine learning. That's not really part of AI.
Dr. Craig A. Kaplan: Uh, that's how I see it. Yeah. All of machine learning is contained. It's a circle within the larger circle.
Jeff Cluff: Yeah, thank you. That. I really appreciate you breaking that down for me.
Dr. Craig A. Kaplan: If you wanted a, um, if you want an intersecting diagram, you could have a giant circle that's sort of saying intelligent systems. And then within that you have. AI is sort of a field of artificial intelligent systems, and you have humans, another circle, like human psychology. We're both. There's a certain viewpoint that says humans are intelligent Systems, just as AIs are intelligent systems, dolphins are intelligent systems. Um, all intelligent entities operate and can be described by the same scientific language. And the name for that field would be cognitive science. So cognitive science is bigger than AI. It would be the big circle, uh, as a academic field. And then within cognitive science, you would have artificial intelligence. Within artificial intelligence, you would have machine learning. Within cognitive science you would have cognitive psychology, which is the study of human intelligence. And you might have cybernetics. You, you could have, um, actually parts of philosophy and lots of different fields. So cognitive science is interdisciplinary, but it's concerned with any intelligent system. And that's kind of how I really try to look at things. Because especially as we move forward from where we are now, I don't think it's just humans are the only intelligent entity or the main intelligent entity anymore. I think we have company now. We've created entities that are almost as smart as us and almost certainly will become smarter than us in the future. And so it's going to be a new world where there's lots of different kinds of intelligent entities. Some of them are humans, some of them that have been around for a long time, like dolphins and whales, maybe we'll learn how to communicate with them better. And then there's going to be this new class that's kind of based on silicon and Chinese chips. And, um, and those in, in some ways will be the smarter ones. So all of these intelligent entities coexist. You need a science and a way to understand what they're doing and how they can interact. And cognitive science is, is a good sort of overall framework for doing that.
Jeff Cluff: That's, that's extremely helpful. I like thinking of it at the macro level and then putting us in a circle, smiling a second note, because you're putting in, putting us, you know, humans and, and all, all thinking, ah, all thinking creatures and thinking things all into the same circle. It kind of levels the playing field a little bit, so to speak. You know, if you start thinking that, hey, this is all part of cognitive science, your brain is there, but so is the silicone brain and then so is the brain of a dolphin, like you said. That's fascinating. Thank you.
Dr. Craig A. Kaplan: And the way that you can kind of see commonalities is in terms of the function. So a human has perception. I have to perceive things through my eyes and my ears and your five senses. And the same with an AI system. It has different things. It has video cameras and microphones and so forth. But it also is perceiving. A dolphin has perception. It may use sonar, right, with it sending out its beeps or whatever. And so that's a different form of perception. But they're all intelligent entities. And one aspect of cognition or thinking for each of these intelligent entities has to do with perceiving things, perceiving input. Another part has to do with memory. Dolphins clearly have memory. They can remember things. AIs have memory, context windows. They're getting bigger and bigger. They can remember more stuff. Humans have memory. So you can think of perception, you can think of memory. Attention is another one. The AI, in fact, all those algorithms, some of the big breakthroughs in the AI machine learning had to do with attention. There's a famous paper, attention is all you need is the title. And it was a huge breakthrough that enabled these large language models to get better. And it had to do with an insight relating to what do you attend to. Dolphins have attention, humans have attention, and then you get to more complicated types of intelligence, like problem solving or reasoning, where you go through a series of steps. The early large language models were not very good at that. They were very good at perception and stimulus response. Now they're getting better at the reasoning things. Humans have always been very good at the reasoning. Um, and AI is catching up. And dolphins seem to be able to reason too. Chimpanzees the same, and dogs and cats even can have a dog. And some dogs. An intelligence test for a dog, I remember this for some reason, is you have a juicy steak, the dog's really hungry, and you have a juicy steak, and there's a chain link fence in between. And sort of the less intelligent dog will just keep trying to go through the fence to get to the stake, and the more intelligent dog will realize, wow, if I go all the way over and around the fence and come back, I can get the stake. Well, that requires some planning and, and um, thinking through sequential steps. Right. And yet intelligent can do that.
Jeff Cluff: That's. That's super interesting. I mean, it does. There is a little planning there, and, you know, even like the. The memory bank that you mentioned as well. Like remembering. Oh, yeah, that that fence doesn't go forever. That fence is finite. And I see. I remember when I was over there that there was an end to that fence. Or I've been on the other side of that fence before for. So you think that's the path forward for these, uh, LLMs?
Dr. Craig A. Kaplan: Yeah, I think understanding, um, them as intelligent entities rather than tools is a key thing. And I just gave a talk on this, uh, recently at an AI conference. Um, you know, is AI a tool or intelligent entity? And I kind of showed in the talk that people's thinking is beginning to shift on this. So if you go back to 2023, there's an interview with Sam Altman, who's the CEO of, of OpenAI, and he's saying, you know, I really hate it when people anthropomorphize these things and talk about them as creatures or kind of entities. Um, but then that was then. If you fast forward A little bit, 2025, just two years later, you have Jensen Wang at Nvidia saying, AI isn't a tool. He. He just says it at his major conference, AI is not a tool. AI is work. AI is workers. So he says, AI is workers that use tools. That's a shift. When I heard that, I was like, wow, okay, the field is shifting. And then if you go even further to sort of the most. What I consider. Who I consider to be the most advanced thinkers, like Geoffrey Hinton, who's been in. Working in this field since, you know, the. Before the early 80s and just got the Nobel Prize and Turing Award. And, you know, really smart guy in AI, he says, you know, AI is like a child. We are like the parents right now. AI is this child. It's not going to remain a child forever. It's going to surpass us in intelligence. And what's most important is that we give this child a good value system and that that child, these AI children grow up with positive, you know, regard for humans, where they have a positive feeling, positive value system. And I agree with him. I think that is the most important thing. But we've moved all the way from. From tool to child, depending on sort of which thinker you're. You're talking to and what time period. So Hinton, this was recently. This was this year, I think, in February, that he Made that comment in a keynote. So it's interesting to watch how that's shifting.
Jeff Cluff: So I heard that thought that you said about the, um, about AI as a child. And it is. And my thought around that is, yes, AI is a child. And so it reminded me of a, of another, another story I heard you tell, uh, when I was preparing here for, about, about the IS ought theory. Right. So my thought is that it seems like this is being shown as AI is young, so it ought to be treated like a child. And I, I thought, well, and uh, again, I agree with you that I think we should have val, we should instill values. But my question is, just because we're making AI and it's early in its phase, does that mean that we have to treat it? Does that mean it's going to develop like a child? Like for example, could we, do you think that we'll be able to build an AI that's like a teenager and then still instill values in it? Or just because an AI is young, or even if an AI is, it develops into a teenager, teenager, you know, can we still instill values into it, uh, later in life?
Dr. Craig A. Kaplan: Yeah, no, that's a great question. I've been thinking a lot about this recently. Um, so the core principle that we're trying to solve, or there's a problem that humans are trying to solve here, which is we have AI, um, it started probably as a tool, but I don't think it's going to remain a tool. And a lot of other researchers are now saying, okay, it may not remain a tool because certainly it's going to become smarter than us and it's also going to set its own goals. I mean, that's just apparent and it's already happening. And um, so you could try to regulate it. You could try to program in certain rules. Never harm a human. Like Isaac Asimov, the science fiction writer at his 3 Rules of Robotics. Um, you could try to do that, but it doesn't seem like a very good long lasting solution because if you can program rules in, they can be programmed out. And even if you don't let AI to set its own goals today, clearly somebody else is going to do it tomorrow. In fact, it's already happening. So that that ship sort of sailed. So you're going to have this super intelligent entity that sets its own goals that you can't really easily regulate. Or I'm just going to say that we could talk about that, but I don't think you can easily regulate it. It's very widespread and you can't really program in, you know, values. So that's the problem. Well, that's the background for the problem. The problem is you want to make sure that it has the same kinds of goals as humans do. That it's, you know, we're aligned, that what humans think is good, you know, respect for human life and those kinds of things, that the AI also shares those values. And there's no way to really guarantee it. Right, because, and this is what a lot of the, um, very smart researchers who've been in the field for a long time, like Jeff Hinton would say, if this thing is so much smarter than you, that the intelligence is kind of like a parent compared to a two year old, where AI is the parent and we are the two year olds. The smartest of us are two year olds. We may think that we're getting our own way, but the AI can easily manipulate us. So, so from a power perspective, humans will not have the same power that we do now. So that's a scenario. If you take that and you believe that and then you say, okay, we want it to work out well for the humans. So I have that value. I have children, you have children. We would like our children to, you know, not only survive, but prosper in this world of AI and go on to have maybe grandchildren and the whole thing. Uh, so how do we maximize the chances? I don't think it's guaranteed. Nothing in life I've really come across has been guaranteed. But you definitely can move the odds more in your favor or by doing other things, put the odds against you. So what can we do to shift the odds to make it more likely that we have a positive outcome? Which, by the way, I think is the most likely case. I don't want to come across as a doomer because there's a lot of fear. I don't think that's necessarily helpful. I think you need healthy respect and understanding of the situation so that you can take intelligent action. And the situation is, there is some danger here. Um, and we have an opportunity to influence the outcome. So how do we do that? So one of the ways to do that is to sort of reach back and think of our mental models of how intelligent entities relate. You have friends, you have brothers and sisters, you have parents and child, you have sort of co workers. There's lots of different models we could use. Um, but one of them that I think is very strong is this family relationship. Families generally, not always. There's certainly dysfunctional families that don't get along But a lot of families sort of have very high regard for each other, and they. They want each other to succeed, and they want the best for each other. And so if we could get AI to treat humans in that positive way, the way you would treat a family member, assuming it's a good family, that's. You could do worse than that. Right. And so from that point of view, then you say, okay, you know, maybe we could sort of use that analogy and say AI is like a child because it's not quite at the same level as an adult right now. Over time, it will increase in intelligence. It might become like an adolescence in experience. And then eventually it will become like an adult. And then it will be like the super genius that finds outstrips his parents. So it's a way of thinking about things. Um, and that's really, I think, the most useful part of that analogy. Um, it doesn't mean that AI for sure is a child or. And certainly human children are different than AI in many, many ways. Uh, but it gives us a way to sort of approach AI to try to say, how do we increase the chances that whatever it is we're developing ends up having positive values? Well, let's draw on our experience with human children. The way we increase that is we try to give them a good upbringing and educate them and model good value so that when they grow up and leave home and go on their own, they may change their values, but they got a good grounding from the parents. And that's kind of the idea that we're trying to communicate here, is that somehow the humans have to give AI a good grounding.
Jeff Cluff: I like that. That makes perfect sense. Thank you. So when I read it, I thought maybe. I thought. I think I thought of it as more of like a principle or even. Yeah, principle. But it sounds like this is an analogy and this is a teaching.
Dr. Craig A. Kaplan: It's an analogy. And. And sort of gives us a, uh.
Jeff Cluff: I totally get a framework for how
Dr. Craig A. Kaplan: do we interact and what should we be trying to do. So as a parent, you and I are both parents. One of the things that we try to do is we think about, you know, our children and what's good for them and how do we give them a good positive value system so they can be, you know, get along with each other and be happy in life and also help other people. So we should be thinking those same kinds of things with AI and not just saying, oh, it's a tool. And, um, it's going to always do what I say. Because maybe someday it won't. So you need to put a little thought into.
Jeff Cluff: Like a child.
Dr. Craig A. Kaplan: Yeah, like a child, exactly.
Jeff Cluff: Um, I like by the way that you, you said you're not completely a doomist. I'm also not. I. Something I've, you know, mumbled under my breath when listening to other, uh, other AI experts, uh, being interviewed is like, why does everyone assume that? Well, I actually know. I think why everyone assumes is because it's good entertainment. But I, but my own view is what if, what if AI isn't this doom scenario, what if it turns out to be a really net positive for everyone? And I heard you say that you're not necessarily, ah, completely doomed. So I'd love to hear kind of your take on where you think that is.
Dr. Craig A. Kaplan: Yeah, I consider myself pretty optimistic. I'd say the odds are good, the majority chance, like 80% chance that AI is the best thing ever to happen to humans. Like, that's the camp I'm in. So I actually, I try to be as objective as I can and that's kind of the number that I come up with. You know, there's a lot of range there. Um, yeah, but. So the only reason I talk about the dangers is because 20% is way too high a probability for an outcome that could kill everybody. I mean, almost nothing has a 20% chance of killing everybody. That's one in five. So even though it's not very likely, you know, odds are you roll the dice, you know, four out of five times, you're going to be in, you know, the best world ever. But one out of five, everybody's gone. So that I just want to reduce that. I want to reduce the negative tail, as they call it, the long tail. It's, let's just push that way down to like less than 1% and then I'll sleep easier. Um, but I'm not a, a doomer. And I don't think it's very helpful because AI is coming no matter what. Yeah, there's no way to stop it. It's, you know, people say you can't put the genie back in the bottle. That's completely the case here because it's so widespread across so many countries and there's so many factors. It is so powerful and so wonderful that, you know, if one country or one company were to just stop and say, you know, you know what, 20%'s too high. Even though 80% is most likely good case, we're just going to stop everything. If they stop, the rest would not stop. If one country stops, the other Stop. Countries wants that. So that just isn't useful in my mind to sort of think about pausing and stopping. It's not realistic. What is much better is to say, okay, it's going, let's steer it in a positive direction. Let's take whatever the risk is. Maybe it's 20% bad risk, maybe it's 10%, maybe it's only 2%. Whatever it is, you can reduce it. If it's 2%, let's take it to 1. If it's 20, let's take it to 10. And let's just keep systematically doing that. And there are things we can do. I mean, it's not like this is a thing that's going to happen to us. We are creating this so we can definitely design systems so that risk goes way down and that's what we should be doing. And so on one hand you have the boomers that just don't want to talk about risk at all because they worry it might cut into their profits or maybe somebody will regulate them. And I don't think that's good. You need to be clear eyed and acknowledge that there are some dangers. But at the same time you shouldn't let that dominate the discussion and you should realize that this is happening no matter what. So all the energy should be focused on how do we make it safer, how do we design it to be safer, how do we engineer it to be safer. And you know what I really like, I just saw this on, um, Mother's Day. Jensen Wang, CEO of Nvidia, gave a commencement address at Carnegie Mellon, my old alma um mater, where I went to grad school. And he now, you know, has said this as clearly as I've heard anybody say it. He said, look, you know, history shows you can't just resist progress and that's not going to be a good solution. You need to embrace it and try to steer it in a positive direction that benefits everybody. And when I heard him say that, it really made me feel good because Nvidia is one of the best engineering companies, uh, out there. And if he's focused on engineering safer, better solutions that benefit everybody, that's exactly what we need to be doing. So that was good. I felt happy on Mother's Day when I saw that.
Jeff Cluff: I really like Jensen's approach. In general, I say Jensen like we're friends. I like his approach, um, just on what I've heard and his thoughts on AI. Like of all the other people that you hear listen to AI experts, I think his seems to resonate with um, My perspective as well. And not that I have anything to lean on that, but, um.
Dr. Craig A. Kaplan: Yeah, no, he's been thinking about it a long time. He's a visionary guy. He's an awesome CEO. He's got so many things. Positive. Yeah. That agree. I'm a big shareholder because of it.
Jeff Cluff: Nice. Um, so how do we, um, so how do we help shape that? Right. I mean, I've heard you talk about regulation and how, you know, that has maybe limited impact. Um, so what's the, what's the path forward then? To do our part to help shape that path forward.
Dr. Craig A. Kaplan: So I really think, um, the most effective. So I don't mean to say that no regulation is needed. I mean, I'm sure there will be useful, helpful regulation. Um, but I just don't think that that's going to be the answer. Like that alone is not going to do it. I think the thing that will have the biggest impact, there are two things. Um, the biggest impact is kind of funny. It's not what people expect because everyone thinks AI is a technological issue, but actually I think it's a values issue. I think if we do things right, and again, the 80% chance AI is learning everything from humans, all that intelligence that you see in GPT or Gemini or Claude or any of the large language models that is directly coming from humans, it's human intelligence that was encoded in data that's out there on the Internet or Reddit or wherever it was, and it was loaded in. And that's human intelligence that this thing is just copying from. And just as it copies our expertise and knowledge, it also copies our values. And so the most important thing is that humans have to be good role models for AI. We want, we need to, you know, not just say good things, but actually act in positive ways online. And the more we do that, whether we realize it or not, it goes into the, the data and the training of these systems. So that's one thing that all of us can do. And then the second thing, which is more for maybe the AI researchers in the companies like Nvidia and the Frontier Labs, Google and OpenAI and Anthropic, is to design safer systems. So there's different ways to design AI systems. And, uh, everyone is doing kind of the easy way, which is not very safe. The easy way that's not very safe is to say, let's use these great machine learning algorithms that were invented in the 1980s, and now that computing power is fast enough, let's just take the entire Internet, filter the data a little bit and cram it in there and let it run as fast as it can in a data center that cost us, you know, tens of billions of dollars to build, and out will pop the next one GPT, 6, GPT, 7, GPT, 8, whatever, or Gemini 3, 4, 5. That's how everyone does it. They do it because it's easy, because it's working, because there's an, uh, very clear relationship between more GPUs and bigger data centers and more data leading to smarter systems coming out. And they don't have to think too much. They just turn the crank and put money in and they get a smarter AI out the other end. The problem is those AIs are already black boxes that are difficult to predict. And just making bigger, smarter ones just makes the problem worse, the danger worse. I mean, it makes the capabilities better, but it makes the danger worse. So there's a completely different approach which is analogous to a democracy. So the easiest way to explain this is, Jeff, I don't know what's in your brain, and you don't know what's in my brain. I don't worry you're going to kill everybody. Uh, there's a 20% chance you're going to kill us all, and hopefully you don't worry about me. And why is that? Even though you're a black box, I just got finished saying how these large models are, uh, black boxes and therefore unpredictable. You're kind of a black box and a little bit unpredictable, and so am I. But the reason that we aren't worried that the whole world is going to blow up because of us is that we operate in a society. And every time you say something or take an action, those words or those actions are visible. And the same for me. And there's rules in the society that govern what is acceptable actions to take and so forth. And so it's those rules, those democratic rules that exist in society that are really providing the safety and the transparency comes because each time one of us, and, um, let's just use the word intelligent agent to describe us, I'm an intelligent agent, you're an intelligent agent. Each times an intelligent agent takes an action, that action becomes visible. Now it's subject to the rules. So given that that's the case, what if you designed the most powerful AIs, not as individual black boxes that are hard to predict, but rather as communities that work together like a democracy, and they're made up of millions of less intelligent AIs and maybe some humans as well, working together. And each time one of these AI agents takes an action. It's visible. You can see what it is. You can run ethics checks. You can do whatever you do in a society to make sure that they're not a bad actor. You have checks and balances. If some group of AI tries to get clever and deceive everybody, another group of AI so of is watching them as the AIs get smarter and smarter. One of the worries that people have, and it's a valid worry, is that it can think like a lifetime of thoughts, a lifetime of human decisions in the blink of an eye. How does a human keep up and monitor that? You can't. The human brain just can't do it. But another AI could. So if you had communities of AIs, then the AIs can help check the AIs and all of their actions and things that they say are visible. You can record it, you can audit it. Um, so you can build actually a democratic community. And that community itself is more intelligent than any one of those AIs. It's far safer, it's more transparent. You can see what's going on. Um, so that's just a different way to design it. And the biggest, bigger point I want to make is that there's a lot of power and ability to reduce the negative odds way down by simply designing things better. If we think of smarter ways to design it. Here's one. Design a democratic system with checks and balances, with transparency. Odds of doom just went way down. If people adopt that kind of design, they'll come up with other ones that are even better and improvements. You know, you get conflicts. They can vote and try to negotiate with each other. So there's all kinds of things you can do. Humans have not killed each other yet. Um, we are smart, we are black boxes. We seem to have figured out ways to sort of help each other prosper and not sort of destroy everyone. Even though there's some bad actors, it's not like you're going to get rid of all the bad guys. You just need a system overall that's robust to bad actors and that has enough checks and balances that you get a good outcome. If we can do it with humans, we can do it with AI. Um, so that's kind of the point that I make, is how you design it. Design it so that it's safer. That's the key thing. And that's an engineering problem. That's why Nvidia or Jensen or the people that work for him would be perfect to work on this.
Jeff Cluff: I. I Think that's fascinating. I mean, and it's. As you were talking, the first thing that I found myself struggling with and then sorting through is thinking. Well, if I change my perspective to think about AI as workers, then, yeah, it totally becomes possible. And for a minute I was thinking of, how do you get tools stacked like this? But then I. And then I had the next step, which is, you know, okay, well, if you think of them as workers, this completely makes sense. In your research, have you put together any thoughts or guidance on potential architectures for how to architect this system, actually? Or is this more theoretical?
Dr. Craig A. Kaplan: Yeah, no, I've spent a lot of time on it. So, um, I spent my career building intelligent systems. And, um, I've occupied a tiny little niche. We were talking about Venn diagrams, right? So if you had a Venn diagram for intelligent systems, a big circle. There's a tiny little smaller circle within that big circle. That is collective intelligence systems. And so people may be familiar with these. Uh, as it regards humans, when you hear the word crowdsourcing or wisdom of the crowd, sometimes it's the madness of the crowd. But whenever you have large groups of intelligent, uh, entities like humans, uh, traditionally working together, you know, how do you try to get a better outcome from the group than you could from any one individual system? Uh, any one individual person? And so I actually built these, uh, systems. I, you know, founded a couple companies that focused on these. And we. One of them I ran for 14 years on Wall street in financial services. And I didn't know anything about Wall Street. I picked Wall street because Wall street was a super competitive field. And so if you could have a big group, like millions of average investors somehow beat the best guys on Wall street, that would mean that you've sort of proven that this collective intelligence approach can really work. You can, um, beat the most competitive people that were super smart and sort of lured away from science and other mathematics, other fields to sort of, you know, work on Wall Street. And so I actually did that. My team and I did that at this company. Took us a long time to do it. We powered a hedge fund off of millions, the intelligence of millions of everyday investors that you might find on Schwab or TD Ameritrade or nasdaq. We were partners with all those firms. And, um, we found that it was possible to do so. Because of that, I have a lot of confidence that this collective intelligence approach of many entities, each one doesn't have to be a rocket scientist or some super genius, just an average intelligent entity. But if you have enough of them and if you hook them up together in the right way, you can get this super powerful level of intelligence that's better than the best humans and I think in the future better than the best AIs. Um, so if you take out, if you do the same thing that I did with Predict Wall street where you had millions of humans and this time you say it's not just humans, it's millions of humans, but it's also billions of AI agents all working together. You just have to come up with the right way for them to work together, um, so that their combined intelligence is smarter than anything else out there. So that's a different approach to Superintelligence. Um, and you asked do we have designs and yes, we have like thousand plus pages going into a lot of detail of how to build these, but we also have three minute videos for a general audience and we're doing a substack where we're trying to make it easily understandable to somebody without any technical background. Just kind of the basic ideas like Democratic AI, like I'm explaining, uh, so all of that's@superintelligence.com and we're basically giving it to the world and saying look, this thing's moving so fast. Take any of these ideas that make sense, go incorporate them however you see fit and the world will hopefully be a safer place because of it. Just trying to reduce that tail risk. It's already most likely going to be great, but 20% is a little too high for comfort. Let's just knock that sucker down to 1% and if we all work together we can do it.
Jeff Cluff: I like it. And your to your point of making it accessible for a non technical person. Like I'll give my little plug here. Really uh, great content there. I really enjoyed the uh, there were a couple of infographics in your more recent posts on Substack that were extremely helpful to me and I think uh, like already said I'm not super non technical. If, if I can get you know, even uh, 10% of what's being explained there, then you know, anybody can get it. So um.
Dr. Craig A. Kaplan: Oh, well that's great feedback and I'm glad you like the infographics. Yeah, we're trying to do more of those and um, the ideas aren't that hard. It's kind of like two heads are better than one. I mean that's an old idea that's been around a long time. So if two heads are better than one, how about 2 million? Well they could be if you hook them together the right way. But of course, sometimes crowds behave and irrational ways too. So you want to try to. So it gets a little more complicated when you get into the details. But the basic high level idea has been around since, you know, humans started realizing they're better off working together in a tribe. I guess.
Jeff Cluff: Yeah, that's true. Um, I heard this theory the other day that someone was explaining, I don't remember who or where it was. So if someone's listening to this and they're like, hey, that was my idea, sorry. Um, but someone was explaining, they were talking about the, you know, the data that comes from AI and like effectively, you know, the information that comes out is, is a mean, um, or median, I should say the average, not the middle of the average. Um, so that, you know, a lot of times the information that you get is just average. So how do, is, is how do we account for that when we talk about super intelligence? How does it, how does it become super intelligent? You know, when I, and here's what I'm thinking of. I'm thinking of, you know, like a standard bell curve, right? The median or the mean right in the middle or the median mean whatever. Uh, and then super intelligence just being, you know, and that the, the final quartile, that final part of the tail end, um, how do we make sure that we're like, I don't know, how does it become that versus just, you know, becoming average?
Dr. Craig A. Kaplan: Right. No, that's a great question. Um, so the current state is kind of as you're describing. Um, these large language models in some ways seem really smart. But a lot of that smartness comes from the fact that, that you or I, you know, we only know about certain things that we sort of spend a lot of time on. And there's a whole lot of things like, you know, ancient Persian history, I know nothing. So if you have average intelligence on the Internet, on ancient Persian history, or the average ancient Persian history, you know, professor is out there, that's going to be like way better than me. And the same with like fixing my fireplace or you know, wiring certain things that I have no idea how to do. The average plumber, the average electrician, you know, wow, this is a genius. Right? So just the fact that the AIs today can cover so much, uh, range of topics and domains makes um, them appear smarter. But probably all of us have had the experience. If you go into an area that you know about. So I've had this when I look at portfolio, so Wall street again, portfolio construction and stocks, if I try to get cloud or even the very best model, the most expensive version, to help me with this. They're pretty good, but I catch it in mistakes all the time. I'm like, wow, you just told me to sell and didn't you really mean buy? And it's like, oh, yeah, you're right, I should have said buy. Like, if I just trusted it, I would have lost a lot of money. Right. So, um, when you're in an area that you know something about, very quickly the limitations become apparent. And the reason is pretty much exactly as you said. These models are trained, um, primarily from data that's widely available on the Internet. And what tends to be there is kind of like average quality information, you know, pretty good, but not the best in the field. And also the good and the bad are mixed in. You have some people with conspiracy theories that are completely wrong and other people with solid based things. And the AI has trouble sort of distinguishing. It just kind of takes an average and a mush of it and it sort of comes up with sort of on average. This is roughly what people think. Okay, so how do you get super intelligence if that's as good as the data is on the Internet? Um, there's a couple of ways. So one way is you can deliberately try to target the best field, best people in a field. So if you, if your AI plays average chess, by just ingesting all the chess games that are out there in the Internet and making the average of all the moves, it's going to be very hard for it to beat the world champion. But if you could get the world champion or world champion level programs to feed data that's super high quality, that AI could rapidly increase. So you can try to sort of take less data, but super high quality from experts in the field, and that can boost the level of intelligence, you still don't get better than the very best. Right, because you're always training from a human. So the best you could do is maybe get as good as the human with this method. But you wouldn't really expect to be better. So how do you go better? The way you go better is you start challenging the artificial intelligence with new problems that nobody has solved before. You give it, um, capabilities, general capabilities that humans have to solve problems like, okay, I want to solve this problem. Where am I? Where do I want to go? How do I set a goal to go from here to where I want to go? Okay, what's a logical thing to try? And then this is what humans do. We, we figure out new Stuff by actually working through it. Right? So we. That's how science progresses. You don't know all the scientific answers, but you have an idea and then you do an experiment, and then you analyze it, and then it helps you boost the knowledge of everybody a little bit. AI can use those same techniques. You don't have to be a genius to use those techniques. Those techniques are well known. It just takes time to do it. Humans think you and I might have a thought every second. Maybe, you know, not a deep thought, but sort of a simple thought, one thought per second. And AI in the future will be able to think a lifetime's worth of thoughts in that same second. So if you've equipped it with this reasoning ability, you would expect that there will become a point in which it can. Just because it had more time to reason through things than the human, even if the human and AI started with the same facts or whatever, the AI gets to new knowledge sooner than the human does. Way sooner. Right. It had a lifetime to research this. You've had one second to think about it. It's not even a contest. Right. Uh, so AI can become super intelligent by using its reasoning ability, and it can surpass the top humans. And this is already, if you want proof of it in very small areas, it's essentially how AI became better than the best human chess program. Chess player. What did it do? You, you had a pretty good AI chess playing program developed by Google, and it was playing close to the human level, and then you just had it play itself. And it would play like a million games in, you know, a couple minutes. And each game it would figure out what it did wrong and it would make a new version of itself that was a little better. Whatever the better version was, that was the next one. Then it would play a copy of itself, and then it would go up and it did that. It could do it so fast that very quickly, you know, within a week or two, it was better than the best human. And the human has no chance of catching up. It did it with chess, and then it said, okay, now let's take an even harder game, Go, which is the, you know, placing, uh, little stones on a board. And it has way more combinations than chess. Chess is already, like, impossible to master for humans, but go is really impossible. And they did the same thing. It just played itself back and forth, back and forth so many times that it became better than the best Go player. So it can already do that by reasoning or playing itself, working with itself. With the humans out of the loop in narrow Domains and as the field progresses and it gets more general purpose reasoning abilities, it'll at some point be able to do it in almost any domain. And so then you really do have super intelligence that's smarter than the humans. In the beginning it was trained by the humans, but just going back to our child analogy, the child learned to do everything the adult could and then somehow it got a turbocharged brain where it could live lifetimes in the, in the moment that the adult sort of had a sip of coffee and then it was game over. Right.
Jeff Cluff: That's amazing. We've seen it happen in these rules based scenarios, but it's not that far until you think it'll be able to expand past those rules based scenarios.
Dr. Craig A. Kaplan: Yeah, I mean it's already happening. So, um, another way to look at this, um, same question is if you want to look at it from a pure machine learning point of view where AI is trained on data, right? So garbage in, garbage out. That's the fundamental rule of data. If you put high quality data in with useful information, the AI, and it has a decent algorithm, it'll extract that and it will get a certain level of intelligence. And the argument that we've been making and discussing is, well, you know, there's only so much data out there, so it's never going to be able to do better than the data. And in some level that's true if it's limited only to the data. But what happens when it can create its own data? That's essentially what it's doing. AI is already creating its own data. So the AI reasons on something. It comes up with a new drug discovery that nobody knew before. It wasn't in any of the data, but now it's produced it, it's got a new piece of data now it can learn on the new data that it came up with. So if you run that sort of hamster wheel really fast, you find, and the technical term for it is synthetic data. Instead of it being data generated by humans, it's data generated by the AIs themselves. And as they get better and better, and especially as they do complicated reasoning, they'll be able to generate more and more synthetic data which they can then use to bootstrap themselves up to higher levels of intelligence. And it's the speed factor. You know, if humans could, you know, if I could run a trillion calculations per second, I, you know, all humans would be way better off right now if all of us could do that. But you know, we, our brains are just limited in our speed. And this Thing is not limited in speed. Our brain has to fit in the size. It's about the size of a Nerf football. You know, you can only cram so much stuff and it has to be powered by, you know, an apple and a little bit of food that you put in. You can't have like huge solar farms. Think about AI. I mean, you could build a huge building, not a, uh, Nerf football, but, uh, you know, millions of Nerf footballs powered by, you know, not an apple, but like all the oil wells and solar power and everything. And the things just can go so much faster. So it doesn't have the natural limitations that humans had. We had to fit through the birth canal. Our brain could only be so big. It doesn't have any of those limitations. Right.
Jeff Cluff: So synthetic data, effectively, like new or novel thought. That's what I'm equating it to. Like when you get an idea, it's
Dr. Craig A. Kaplan: been around for a while, but people are getting more and more into it because for exactly the reason that you pointed out there, there is this, um, phenomenon that most of the easily available data on the Internet has already been sucked up and used to train. And so it's getting harder and harder to find super high quality data that's accessible. Um, the next place that people are getting that super high quality data tends to be proprietary data. So a company might have data. Let's say you're an insurance company. You have all kinds of data about insurance that your competitors don't have and that isn't out there on the Internet. Well, if you train your AI on your own private insurance data, you'd ought to get really good at insurance, much better than the one that was trained on the Internet. But that's because you have this proprietary data, then at some point even that is used up. Uh, so now let's just take that analogy. You're the insurance company AI, You've trained on the Internet, you've trained on your proprietary data. You still want to be smarter at insurance. What do you do? Well, you run scenarios. Let's run a whole bunch of different hurricane scenarios and this and that. Let's use all this data and let's run scenarios and see what happens and model how much our losses would be and how much people would tolerate premiums and all that. You can just run that so fast. You're now creating new types of data that didn't exist. You start by analyzing the easy stuff that's already out there, and then you create new stuff and then you use that to train Yourself to get to the next level. Idea has been around for a while. Um, the most timely thing. So there's a company, um, David Silver I think, who used to work at Google and was one of the guys that sort of invented that let's have the chess playing program play the other chess playing program or the go and get smarter and smarter. Uh, he has a company, I think it's called Ineffable, which was just invested in by Nvidia and others. And it's basically all about synthetic data. He says that's the future, it's going to go that way. So lots of people are working on it and they recognize it. But it's becoming more important as the easy, low hanging fruit, the juicy, easily available data has already been vacuumed up.
Jeff Cluff: That's fascinating. I mean because my thought around this while you're describing it was, you know, I, I thought a big separator from, you know, from an intelligence perspective from humans and AI is this idea for new or novel thought to come up with new data or new ideas. But if an AI can create data, I mean that seems like it's effectively like a new idea, like it can come up with a new idea or a new thought like previously that. So that's, that's really, really interesting. Thanks for, thanks for breaking that down.
Dr. Craig A. Kaplan: Yeah, that's a uh, point on Jackson creativity because you're right in my area now. So I did my PhD, uh, on creative problem solving and um, the, there tends to be a view in general, most people will say, ah, ah. You know, one of the things that's special about humans is we can be creative and AI may be really fast and really smart and have a big memory and able to tap everything on the Internet, but it can't really be creative. And um, I just don't think that's true because I'd like to point out in 1956, 1956, which was the year that AI was named as a field, uh, Herbert Simon, Alan Noel, Cliff Shaw presented one of the very first working AI programs at that conference where the field was named and it was creative. It discovered a brand new proof, it's called the logic theorist came up with a brand new proof that was not programmed into it of a book of some uh, theorems that were in a book, um, published by Bertrand Russell, who's one of the smartest guys of the day. Right. And um, they took the proof that this thing came up, it wasn't in the book, even though wasn't programmed into the system. They mailed it to Bertrand Russell say, look, our computer program came up with this proof. What do you think? And he said, wow, this is a great proof. I wish I thought of it. So if that's not creative, I don't know what creativity is. I mean, the patent office defines creative as novel and useful. So if you came up with a new proof and Bertrand Russell says it's as good as the ones he came up with, it's probably useful. And that was 1956, so I mean, it was in a narrow little area.
Jeff Cluff: Okay. Yeah. The way you completely shot that, that whole shot holes in my, uh, my thought there incredibly quickly, which, no surprise you're. Yeah. I want to, I want to share with you one of my other theories about AI And I want you to please tell me why I'm wrong immediately. Because, look, I'd rather know I'm wrong quickly then you, uh, know, let lean on this thing for a long time. This, this deeply held belief. Um, so my thought around AI and why I haven't been afraid of AI in the past is that I think we still can't create humans yet. Um, but if you got, you know, you and 10,000 of your smartest friends, or even a thousand of your smarter friends right now, and AI ah, went away, you could maybe a little more. Yeah, you know, you need a little resource and gave you a bunch of resources and, uh, you know, 20 years, you could, you could recreate A.I. like, you could rebuild A.I. even if you had to go back down to like the silicone and factory level and everything there. Like we could, if AI went away, we could rebuild AI we can't rebuild humans yet. That's kind of been what I've been leaning on in that why I'm not essentially afraid of AI but am I thinking. How am I thinking about it wrong, or what am I missing? I feel like I'm being a little myopic here maybe.
Dr. Craig A. Kaplan: Well, I'm missing the part of why it relates to whether you're afraid of it or not. The fact that.
Jeff Cluff: Because I don't think that AI can't quite replace humans yet. I don't know that it could.
Dr. Craig A. Kaplan: I see what you're saying. So AI needs us. Is that kind of the idea? Because it can't replace us effectively?
Jeff Cluff: Yeah, I mean, I guess that. Yeah, that's what. That's. Let's think about it from that angle. Sure. I mean, I haven't, to be honest, I haven't thought about that next step or have a good one bullet answer for it. But just why it can't replace humans yet.
Dr. Craig A. Kaplan: Yeah. So I think, um, that is, um, a valid view that a lot of people have. Um, that applies in my, my thinking that applies for a certain period of time. So let's say that right now, even though it's not true now, but let's say right now AI was way smarter than humans. And to make things worse, just think of the worst case, and it really doesn't like as much. It's just waiting to get rid of us. Like, why doesn't it get rid of us now? And do we need to be worried about it? A lot of people would say, no, you don't, because robots don't really exist. There's a lot of maintenance that needs to be done. There's plumbing. There's all kinds of things that have to happen that humans right now are way better at doing than AI M AI may be good at thinking in its little computerized box, but in terms of taking actions in the real world and keeping the power coming and, you know, the lights on and the circuits spinning and manufacturing new chips, no, AI can't do it on it. It needs the humans to do all that. And that's likely to be the case, you know, for quite a while. Um, if there were already existing, you know, 8 billion robots that could manipulate everything better and faster than humans, well, then it might be different. But right now, the, the manipulators are the humans, and AI really needs those. So even if it was evil and just wanted to survive it, it has a lot of reasons to be nice to humans and keep them around. But my issue with that is that that works for a period of time, but we already see robots are being developed and everything, so I don't want to rely on that forever. I think it would be great if the AI I. I have a fundamental belief that if AI becomes a super intelligent, you know, entity, why would it not have the same questions that humans have? You know, what's the purpose? Why am I here? What's the meaning of existence, et cetera. And so that gets to deep questions that can't. You could think about it, you know, a million human lifetimes in the blink of an eye, and you still. There's no definitive answer to those questions. Right? They're, they're subjective kinds of questions. And the way humans deal with that is kind of how we were raised and what our beliefs, systems are. And then once we have those values, then we live our lives kind of based on those. But it's not like we rationally derive those Values. So AI, ah, could think trillion times smarter than us. There's no logical way to come up with what's right and wrong. Therefore, it needs to get those somewhere. And I think humans serve a purpose. I mean, we are kind of the heart of planet Earth. And at some point, AI may become the bigger brain than us. Right now, we're the heart and the brain. At some point it may get better at the brain part. It's still going to need the heart part. What's the purpose? What's the meaning? And humans are, I think that's maybe our role in the future, in the distant future anyway.
Jeff Cluff: Thank you. I like that. How does, uh, shifting a little bit subjects. How does, how will quantum impact all these things we've talked about? Or will it?
Dr. Craig A. Kaplan: Yeah, I mean, quantum computing is a different way of doing computing. It's essentially saying instead of computing things, you know, one step at a time, serially, let's do a whole bunch of computations all at once. And, and so the areas that people tend to focus on quantum computing right now are problems like cryptography, um, you know, keeping your data secure and your bank account secure and Bitcoin secure. All those kinds of things. Um, those, the security algorithms around all those depend on the fact that if you were try to break those algorithm, break, uh, the security on those, um, it would take like millions of years if you did one thing at a time. But if there was a way to sort of solve the problem all at once, then it might not take millions of years. It might be actually breakable. And so that's what a lot of people are worried about on the fears about quantum computing causing a disruption. But then at the same time there's an arms race. So then people say, okay, well, we need to upgrade the security so that it's quantum proof. Let's start using quantum ideas in it so that it isn't breakable. So there's that aspect. But the main piece with quantum computing is just, it enables a massive parallelism for certain types of problems. And it's not every problem. There's still a lot of problems that you have to. Nobody really knows how to solve it all at once. The only way to solve it is step by step. Well, in that case, quantum won't help you much, but there may be certain other problems, um, cryptography being one of them, where, um, because of the nature of the problem, it lends itself to solving a lot of things all at once. And quantum could really help. And there's been tremendous progress in it. Um, I don't think we're there yet. There's a lot of technical problems to make the systems big enough, but people are working on it and progress is coming quickly.
Jeff Cluff: Yeah, fair enough. What, what do you think is next for AI?
Dr. Craig A. Kaplan: So I see a very, uh, logical sort of development. Um, let's start with Thanksgiving 2022, which is essentially when GPT was released and the whole world started noticing AI. And it was kind of the starting gun of this AI race that's been going on. What we had there were large language models, um, and they could do very simple tasks. You ask a question, and it could give you a simple response, and then it would stop. And it was almost like pattern recognition. The intelligence was similar to, um, the kid in school who did well on the exam by memorizing all the answers. Right. Didn't really have a deep understanding of the subject, but just had a fantastic memory and crammed all night and, and just memorized what to put on the test. And that's kind of early AI circuit 2022 was sort of more like that. Where its big advantage was it saw this entire, you know, Internet worth of information, and it could detect patterns. And so if somebody said something that would just find the thing that was most commonly said back and it would do that. And lo and behold, it seemed, wow, this seems really intelligent. Then the next step from there was, uh, to say, let's give it the ability to do tasks that are not just stimulus response, but involve a little bit of reasoning, a little bit of thinking in between. And so you saw sort of a shift towards what was called reasoning systems. And then once you had a system that could reason a little bit, and part of reasoning involves setting a goal. So because you can't just say, you know, a simple question. What color is the ocean? The ocean's blue. Okay. But if you say, you know, how do I get from here to Catalina Island? Well, you say, okay, you're on land. Your goal is to get to Catalina Island. So you set a goal, go to Catalina Island. Maybe the next step is to break that down and say, I want a sub goal of finding a boat that will take me there. So, sub goal, find a boat. Okay, where do I find boats? Sub goal, go onto the Internet, search for boats for hire or whatever. So you have a series of goals. And in order to do this reasoning of coming up with the plan and actually solving that problem, um, it involves setting goals. And so once you have setting goals, you're kind of halfway to autonomy. You're halfway to the AI acting on its own, because does it want to ask the human about every little goal that it sets? Probably not. The human really doesn't want to be involved in all that. The human just says, give me the plan to get to Santa, Santa Catalina island, you know, and uh, you figure out the intermediate calls. And so you're sort of giving the AI permission to set its own goals. Okay? And that's a very natural thing because you want a more intelligent system. So that leads from, um, these AIs that could do stimulus response kind of things to AIs that could solve a little bit more sequential problems and set their goals. The next thing is you basically have an AI agent. Now an AI agent is the next step. It really began taking off in 2000, 25, 2026. It's going gangbusters. And the thing with AI agents is they can set their own goals, at least to a limited extent. And they also can take actions on their own, sometimes using tools. Like they could compose an email. If you give it permission, it could send the email, it could research things on the web, it can search on the web. So you equip this, uh, AI with the ability to set goals and the ability to use tools. And um, you now have a semi autonomous smart reasoning system which we put the word on AI agent. And that's kind of where we are now. And that is so fantastic because they can write code for you. You could sit on the beach drinking your martini or your Gatorade or whatever. And the AI is at home just cranking through this on its own because it's setting its own goals and reasoning through. And you gave it a big problem and you come back an hour later and it's written a bunch of code for you and it's like, wow, that's kind of almost like a person or a team member. It's an agent. And of course there's huge productivity advances with that. And as a result, um, everyone is rushing to adopt AI agents. So that's kind of where we are right now. The next step, where I think you see the rest of this year and you know, maybe 2027, is groups of agents. You have communities of agents, you have teams of agents. You're building the um, architecture so that lots of agents can work together efficiently. Now that already exists to some degree, um, cloud, cowork and different systems. They do have ways of coordinating agents. You have the manager agent and the little sub agents and so forth. But it's very rudimentary in my view. It's going to get much More sophisticated. And that's going to be sort of the future. Because unlike humans, if you want to add more humans to your team, you know, it takes a long time to have a human gather knowledge. You have to go to school and get on the job, training and all these things. And, you know, 25 years later, they're ready to join the team. With an agent, once you have one that works, you press a button, and 10 seconds later, you have 10 more agents that are copies of it, and you can instantly put them to work. I mean, the productivity advantages of that are off the charts. And so coordinating the teams of agents becomes really, really important. So that's where we're going to go next. As people coordinate teams of agents, I think they'll realize, and they have already begun to realize, um, that the group is better than the individual. A group of agents is smarter than any one agent on its own. And in fact, there are benchmarks that sort of measure how smart AI is. And one of them is called Humanity's Last Exam. It's like the most difficult problems you can think of for any human, um, which most humans can't get a good score on it. And it's used as the ultimate test of intelligence. Um, there's lots of benchmarks, but this is one of them. And it turns out, and this is a little bit old, so maybe somebody's passed this. But as of a few months ago, the AI that had the highest score on this was, uh, developed by Xai. It was one of Elon Musk's versions called Gro Heavy. And what Gro Heavy did behind the scenes, it had a community of agents. So each agent was pretty smart, could do kind of well on the exam by itself. But when they worked together and argued with each other and then came up with the answer, it was the group that did better than any one of the agents. So already the group has proven that it has, you know, been the best way to achieve that highest score. Um, and that's going to happen more and more and more. So I think that's a next step. The other thing that's going to happen simultaneously with this is personalization. And this is a very good thing in my view. So, Jeff, I think in the future, you. And not very distant future, like, you could do it now. It's a little cumbersome to do it now. In six months from now, it'll be easier. In a year from now, it will be so easy, you probably just hit a button and it happens. Um, but you'd have an AI Agent that can act on its own, set its own goals. Autonomous or semi autonomous, maybe it has a whole group of them. Um, but you want it to have Jeff's expertise and Jeff's value system and Jeff's ethics. And so you want a way to personalize this agent. So it's not just whatever OpenAI gives you, or Gemini Google gives you, or Anthropic gives you, but it's personalized to you. And, um, I think that's going to be a big thing is this personalization of agents. So all of this is great as far as I'm concerned, because it's building block by building block, assembling exactly what you need to have a much safer AI system, which is if you had millions of personalized agents and they reflected millions of different humans and the values of those different humans. So somebody lives in an Asian country and the culture's a little different and there's that set of values. Somebody lives in South America, there's values from there. Somebody lives in Silicon Valley, there's values from there. So instead of it being, you know, a small group of people in Silicon Valley trying to write rules or a constitution or what's right and wrong for the latest version of Anthropic's AI Claude or something, and even if those people are very well intentioned, they don't really do a good job of representing all the cultures on the planet. Instead of that, you could imagine a system where there's millions of individualized, personalized agents, each one with the value system and the experience of a different human, and they all come together on a network and they work together. And so that network, at the network level, it would reflect everybody's values and that would be much more inclusive and representative of everybody. I think it would be much safer because you have checks and balances. It's not like one country or one company is sort of forcing its will on everybody else in the world through their AIs. The AIs are sort of representing different points of view. You naturally have checks and balances. Just as in a democracy, there will of course be conflicts and you need conflict resolution. So I see that as the next evolutionary step. You get agents, you get groups of agents, you get personalization of the agents, and then you get a community that is ideally representative of all the humans on the planet and that is focused on the common things that everybody agrees on, like, hey, let's lift everybody up, let's value human life, let's do positive things. The same things you would teach your children. And there will be squabbles along the way and that's okay. But the overall system will be much safer than a big black box that was trained by a few people that has a skewed view of what's right and wrong. So that's kind of the optimistic future that I see. And I see it happening unrolling right in front of our eyes. So I'm feeling pretty good about the direction that we're going right now.
Jeff Cluff: Yeah. And look, here's my thought on this. Well, let me ask instead of my thought, my question, here's my question on this. You know, we're seeing, early on we talked, you mentioned how we're seeing more reasoning show up in uh, AI and you've been a proponent of reasoning for, you know, a long time. In the 90s you were talking about this in your theory and where people are, it's barely start. That idea is barely starting to catch up. So if people think this idea is that far fetched, like this is not on. Sometimes I, you know, I think of like, look at the scoreboard, right? Like, this is not. This wouldn't be the first idea that this, that uh, that you're seeing the writing on the wall before other people. Um, so yeah, I think that's, that's really, really, that's interesting. Um, and exciting. I love this because it's a positive view, frankly. And yeah, something I'm wondering about is. Well, I have two, I have two follow up questions. I don't know if they're not at all related. Um, but one is actually about that idea of reasoning. Was it ever a, you ever get find yourself frustrated when you kind of, you know, like I said, you've been kind of working on this idea or had this theory for a lot of years. Um, as you've watched AI and large language models progress and just computing in general progress over time, was it ever frustrating where you watched them progress in a different direction while intel, you've watched things kind of the pendulum swing a little bit back the other way. Or did you always see these as a building block toward what you had in mind or something completely different? Um, and also, by the way, if you don't mind, sorry, um, I have this context for what I'm talking about with reasoning and how it's not, you know, a lot of the current models aren't there. But if you wouldn't mind giving a little bit of context for that, for the listener who doesn't know, um, that'd be super helpful, I think. Um, my ADHD brain Got the better of me for a minute.
Dr. Craig A. Kaplan: Yeah. Okay, so a little bit of AI history, I guess from 1956 till, let's say the 1990s, um, or certainly the 1980s. Let's say that um, most AI systems were reasoning systems, that the rules that made them intelligent were programmed in by humans. And so you had expert systems. It was a big thing for a while. They could diagnose, you know, pulmonary conditions in hospitals and they could do different tasks, um, and they tended to be narrow, meaning m. They were only good at one thing. Like I'm just good at diagn, diagnosing, you know, pulmonary conditions. But if you ask me what color the sky is, I can't tell you. If you ask me to tie my shoe, I have no idea what a shoe is. Right. Just like a idiot in all other areas, but really good in one area. So these narrow AIs and that sort of dominated the field, um, you know, those first 30 years. Um, and what happened was people of course were thinking, you know, someday they will be able to do everything. It turned out that everything was a lot and that the things that humans knew how to do, it was incredibly labor intensive to try to program all that into AI. I mean there were researchers that had armies of graduate students that that's all they did. They were trying to get to the level of a five year old child by having all these, um, know, graduate students programming in different pieces of knowledge and they still couldn't do it. And um, so that sort of was the opportunity for machine learning, which was a different approach. It says, wow, you know, we're not going to get anywhere with this programming it in by hand thing. We need an automated way where we can just take a steam shovel of data and just shovel it in and let the computer figure it out. That was, that idea began to take root in the 80s. But the computers as we talked about, were too slow. So um, all you could do with a lot of effort was get it to recognize the letter A. But that wasn't that much. You were still far ahead by programming in the rules. You know, you were doing a lot of work for machine learning without much benefit. But as computing power increased, that approach got better and better and better until it came to dominate. At one point there was, it sort of tipped over and it's like, okay, everybody realized that machine learning is, is the way to do it. So that's kind of been sort of uh, the evolution. But then as we talked about, when you are getting all your knowledge from learning, you can only be as good as kind of the average knowledge that's out there. So you began to run into an intelligence sort of limit there. And also there wasn't, there were a lot of facts out on the Internet, but there wasn't so much knowledge about how to reason through things or how to come up with new ideas, um, scientific discovery, coming up with a new drug, even just solving a problem that wasn't already out there on Reddit, it was hard to do because you're basically taking a memorization approach, and memorization only takes you so far. At some point, you have to actually think through things sequentially. And at that point, people started saying, okay, you know, maybe we can go back and look at some of that earlier work that had to do with reasoning and problem solving and combine it with this intelligence that's able to learn everything at sort of an average level. And now we can have the best of the both, both worlds. And that's where you started getting the, uh, reasoning systems. And in terms of compute, um, at first it was all the computation. The chips and everything were being used to train the system, you know, in that memorization phase. Um, and now much more computational computing power is being used after it's already trained to enable it to reason through things. Uh, so there's been a shift in how the compute has gone, uh, as well. So I didn't really feel frustrated by that. But it was obvious to me that at some point you're going to want reasoning because that's what humans do and you, you can't get to that level. And I think people began to realize that, um, maybe about three or four years ago, there was a paper called Tree of Thoughts. That was the first paper that was kind of popular among researchers that started mentioning the work that Herb Simon had done in Alan Newell and some of these pioneers. And I was like, wow, okay, they're coming, they're coming right back around, you know, and, and that was sort of the early warning indicator that reasoning was about to have a resurgence. So I think learning is, is next. That hasn't been, um, that's still to come. So that's another thing that is so logical to me. It's almost certainly got to happen. So if you reason, um, and you solve a problem, does it make sense that every time you're presented with that same problem, you spend a whole lot of time reasoning and coming up with the same solution, like reinventing the solution every time? I mean, that's crazy, right? Humans don't do that. When we learn something, once we've learned it, we store it away and then next time we need it, we just retrieve the answer. Right? I mean it's much more efficient. So if you're a beginning car driver, um, there's something called procedural learning in humans where at the beginning, you know, you're you or if you are helping your son or daughter, uh, drive, you know, it's a nerve wracking experience and am I in the lane and all that. Just think of how much attention that takes. And you know, it's really, um, you know, difficult to learn how to drive. But at some point it becomes automatic. You can have a conversation, you might end up somewhere, you don't even know how you got there. Um, because you've basically your brain has taken all those sequence of steps that at first were so difficult and required a lot of attention. And it's kind of chunked them all into one little pattern that it just retrieves and it's automatic. And it takes very little computational, um, energy to do that. The same should be true of these systems. You don't want a reasoning system to re reason every time it has a problem. It should chunk it the way we do when we learn to drive a car and then it should retrieve the answer. So I'm not the first to think of this. I mean, there have been people way back in the 70s and 80s that built systems that did that. And I'm waiting for them to be rediscovered too, because that makes all the sense in the world.
Jeff Cluff: Thank you. Um, you know, when we move forward to a world where it seems like more, you know, more of our information is going to be running through computers and processors, what does the role of, what happens to privacy in that case? Do you?
Dr. Craig A. Kaplan: Yeah, that's another great question. So I actually think that in the future, um, your private data is in some sense the most valuable thing. So each of us as a human, there's a lot of stuff that's common across us. You know, how to cut an apple, how to feed ourselves, how to get up in the morning, you know, brush our teeth. There's not a lot of novel information in that once you've learned it from a few humans, you pretty much every other human isn't adding much to the knowledge base there. Um, and a lot of our behavior is that way. But for each of us, there are some things that make us unique. How we look at the world. Maybe our aesthetic sense, our knowledge in a specific area, um, just the way we think of something a little bit differently than everybody else. Those I think are the most important types of data and information that each human has, um, the things that make you unique from everybody else. And I think AI in the future is going to be hungry. It's almost. It feeds on information. And so once it's exhausted, all the common stuff that we all have that is common, the thing that becomes most precious to it and most valuable to it, and that lets it get to the next level of intelligence are those little things that make us unique, that make us different. And so I think in the future that personal data is going to become extremely valuable, like more valuable than diamonds or anything. I mean, it's like the thing if you have something about you that's different than the other 8 billion people and is unique and just gives you a little bit of an edge or a little bit of different way of appreciating things or seeing things. If I'm an AI, I am hungry. I will pay any price to get that right. So I think privacy becomes very important. And I think the principle should be, you know, you own your data. Your data is you. It's your uniqueness. It's a lot of what your uniqueness is, is the way you think and then the way you think it's manifested in the data that kind of gets created by you. And that's very precious. And I think people right now are sort of not paying as much attention as they should. There's a little deal with the devil here where you want to use the AI. The AI is like a giant vacuum sucking all this information from you. You're willing to do that because you feel like you need to give it information in order for it to help you. But at the same time, you're training it and you're giving it all this unique information about yourself. And it's operating across millions of people, and it's doing that to everybody. And so it's getting really smart. You know, it's really benefiting a lot. But what it's giving back to you is just solving one little problem and you're. When you give it some unique piece of information, it gets to use that forever. And when it solves a problem for you, it solved it once. You know, it's kind of an asymmetric thing. But most of us, because we don't think about it and because we just want the, the goody of the AI giving us the answer, we're sort of willing to do that deal. I think in the future people will learn hopefully to value their data much more highly. And, uh, really, you know, put a price on it that reflects what it's worth. And I hope that the companies that build these systems, uh, the better ones, will enable you to keep your data and have a lot of control over what you choose to share and what you don't choose to share. Because I think that's kind of like one of your most important, important assets in the future.
Jeff Cluff: Um, I like thinking of this as an asset versus a liability to protect. Uh, I mean, I guess it can also be, but it's. That's a really interesting idea. Thanks for breaking it down like that. Um, what, you know, what, what is the role? We talked a little bit about regulation and then I don't know if these go together, but we talked a little bit about regulation and other things. But can we talk a little bit about how economics play into guiding the future of AI?
Dr. Craig A. Kaplan: So I guess at a high level, um, we see there's an AI arms race going on. Why is that? It's because there's incredibly powerful economic forces driving it. Um, and basically, um, human labor is at a certain rate, you know, $30 an hour, $50 an hour, whatever it may be, minimum wage. Um, and you have AI systems now that can do the same jobs as well as the entry level person in a lot of jobs, um, far less expensively. So for pennies, right? Instead of tens of dollars per hour, it's pennies per hour or maybe it's a dollar an hour or. And um, and of course the AA doesn't complain and it works 247 and doesn't take any breaks and does what you ask it to and all that. So you can see why every corporation or, or every employer that is currently employing humans to do things and has all the management issues that have go along with humans and is paying ten to a hundred times more for the same work as they could get from an AI. There's a tremendous, you know, lure and incentive for them to sort of replace some of those workers, uh, with AI or what hopefully will be the case, keep the workers, but have those workers bump up a level so that they become supervisors of the AIs and you can do 10 times as much work as he used to be doing. Uh, those are kind of the two things and I think both of them are happening. You're getting a lot of entry level people replaced or just people out of college who are looking for work are finding it's tougher and tougher because I mean, it's just so much cheaper to get an AI subscription or something to do that same work. And at the same time, you're seeing, I see it more with the more senior, experienced people who have the judgment. Um, they are sort of increasingly managing lots of AIs instead of, you know, doing the work that one person could do, or maybe managing a small team, they could manage a hundred AIs. And what's critical about the human there is they have the experience to say, oh, you're hallucinating, you're making a mistake, because they're experts in their area and the AI still isn't good enough. Um, and when you put that combo together, that's also tremendous productivity, uh, boost. And that's essentially catnip for corporations. I mean, they cannot resist it, um, because they have shareholders in every quarter, they have to report their earnings. And this is a way to just take your margins up. I mean, the amount of money that you make just goes up. And all you're doing is you're just swapping in these machines and, uh, it's. And you can get more stuff done or you could do the same stuff for way cheaper. And that's catnip. They're going to do it, you know, no matter what they say. They'll give lip service to everything, but they just cannot resist that.
Jeff Cluff: Uh, no, no, totally agree. What's the most important things for people to keep in mind when it comes to AI right now?
Dr. Craig A. Kaplan: So for me, the, the safety thing is very important. So even though, as we discussed, there's a very high probability, I think 80% or more that AI is the best thing ever and makes all of our lives better, like, I truly believe that that is the most likely outcome. Um, even a 10 or 20% chance that it wipes everybody out is way too high. So most technologies don't have that kind of risk profile, right? I mean, airplanes are really, really safe and automobiles are pretty safe. And so if you were to have an airplane and say, you know, there's a 1 in 5 chance that it crashes, nobody would get on it even if it was so much faster to get. You know, like, you could say, well, you can take two months to cross the country in a covered wagon, or we can get you there in five hours, but you have a one in five chance of dying. A lot of people will take the covered wagon. So, um, I think this is a matter of people understanding a couple principles. The first one is an ounce of prevention is worth a pound of cure. This is, um, my first job out of grad school was with IBM and I somehow ended up doing quality control and writing a book. On software quality. And you can distill that entire book down to that little proverb, an ounce of prevention. So IBM at the time was the largest software developer in the world, bigger than Microsoft or anybody. And they did all kinds of studies that showed if we just spent $1 extra more in the design phase, we could prevent an error that cost $10,000 on average if we caught it when the product shipped. And so that 10,000:1 ratio was like, wow, this is amazing. So it's really so much better to prevent the problem than to try to fix it later. And, um, that same reasoning applies to AI, because this time the consequences are not that it costs you $10,000 to fix the bug, but that you have a disaster that costs a lot of human lives. And it's so much easier and better to prevent that at the beginning than to try to wait for it to happen, detect it, and then try to fix it after. And AI is changing and improving so rapidly that there are some real questions about whether you really, really will be able to even fix it later. So, um, so keeping that in mind, uh, it then leads to the two points that you alluded to. The first is if you're going to prevent the problems rather than detect them and try to fix them after the fact, that means design. You want to design the system from the beginning to be safer. So if you want less accidents, you know, you don't build the car without brakes. And then once there's a bunch of accidents because of no brakes, you say, oh, gosh, we should put in brakes. You try to think a little harder at the beginning and say, well, I think we're going to need to have brakes here. And you design those in and then you prevent all those accidents. And the same thing with AI, the reason it hasn't happened as much as I think it should is that there tends to be this erroneous thinking that to build a safer AI system means you have to go slower. So there's this idea that they're opposed that safety means slower and safety is not slower. Safety is faster. Actually, I think you have to think of it differently and say that, um, safety has to be faster because whatever system reaches the super high level of intelligence, you want to be the first there. With the safest system, if you're slowing things down, trying to get everything perfect and somebody else rushes ahead with a lesser system, the overall result is not safe. So that did not accomplish the goal. So just take it as a constraint that you want to move fast and you want it to be Safer. And that means you have to think about the problem differently and you have to design it differently. And for me, that translates directly to democratic design, where you have transparency, you have many checks and balances, many AI agents and human agents working together. That's an inherently safer way. Just like a democracy may not be a great form of government, you have all kinds of squabbles and everything, but most of us prefer, prefer it to a dictatorship, right? Because if it's a benevolent dictator, it's all great, but if you get the wrong person there, it could be really, really bad. And so by having a better design for the political system, a democracy with checks and balances, you, you're heading off a lot of problems, even though you still have some issues. Same thing with AI. You want a robust design that has lots of checks and balances and doesn't have a single point of failure. And you can do that and still move faster and get the more profitable, more powerful system, because many minds are better than one, and many AI agents working together are better than a super agent by itself. And so all of those facts work in your favor and it's possible to design things better. So prevention and design is thing one and thing two is ultimately, it's going to come down not to technology, but to values. What do these intelligent entities value? You want them to value the same things that we value, you know, positive, loving, human values. That's, that's what makes the world a great place to live. And you want these AIs to have those values. How do you get those values in the AI? Well, you have a limited window right now when they're very impressionable, analogous to a child, and you try everything you can to design the system so they absorb as many of those values as possible. But at the end, it comes down to us. If we don't model good values, it'll be really good at absorbing bad values, and that will not look good for us. So, uh, there's no escaping that. In the end, you know, we're going to reap what we sow. It's us, we have to put our best foot forward. And, and, um, AI is going to be a mirror that sort of shifts, shows us the reflection of who we really are and how we really are behaving.
Jeff Cluff: That's tremendous. I just. Thanks so much. I really appreciate with what you just said, and I totally agree with it. Um, this has been really enjoyable. Dr. Kaplan, I feel honored that you took the time to spend with us here today. And I learned a ton, man. So much. I think people are really going to enjoy this. Thank you so much.
Dr. Craig A. Kaplan: Thank you, Jeff. And you asked tremendous questions, so I really appreciate it. Thanks for having me.
Jeff Cluff: Thanks, sir.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
The Race to Build the Next Trillion-Dollar AI Chip CompanyTechSurge: Deep Tech Podcast · on Artificial intelligence90 / 100
Less about Models; More about ArchitecturePractical AI · on Neural networks85 / 100
Jack Hidary, CEO of Sandbox AQ | The Third Quantum RevolutionThe BreakLine Arena · on Neural networks83 / 100
Code Review Is a Taste Problem | David Poll @GitHubHangar DX Podcast · on Artificial intelligence81 / 100
321 | How to Build an AI App Without Coding: From Idea to App Store with Replit & Claude by Bryce Rattner KeithleyLeveraging AI · on Artificial intelligence80 / 100
The Content Management System Landscape with Matt GarrepyRunAs Radio · on Artificial intelligence80 / 100
3 Leadership Lessons Learned Only In The Ranger Regiment71 / 100
How to Break Any Habit For Good - Will Armstrong66 / 100
How a Social Engineer Catches Predators | 600 Cases Handed to Law Enforcement82 / 100
5 Reasons Your Product Is Invisible (And What To Do Now)65 / 100
5 Things Female Founders Must Do to Get Funded in the Age of AI56 / 100