The "AI Bubble" Bubble | EP.51

Here's today's question, is all this talk of an AI bubble itself a bubble? For the past few years, we've heard a steady drum beat that our official intelligence is overheated. Billions of dollars are flowing into data centers, advanced chips, and frontier models. Nvidia is investing in companies that turn around and use that capital to buy Nvidia chips. Cyber-skillers are committing to massive multi-billion dollar cat-backs. From now outside, it can look circular and speculative. But at the same time, something measurable is happening beneath the headlines. Systems that struggle with meaningful software tasks just two years ago are now passing serious engineering evaluations. Enterprise adoption has moved from experimentation to operational deployment. So which signal matters more? The scale of the capital or the slope of the capabilities curve? Are we watching the early stages of the most important technology shift of our lives? Or living a side of classic hype bubble that will burst soon? Today we'll give you our perspective from inside the AI industry, not as spectators, but as builders deploying these systems with real customers. What does it actually feel like to operate an AI right now? What concrete evidence suggests durable value creation rather than financial excess? And what specific developments over the next six to 12 months would cause us to change our minds? Let's get into it. So let me kick this off with a little bit of discussion from my perspective. There's been a lot of talk about a bubble within artificial intelligence, I think for the better part of two years. A lot of this being driven by just the massive cash infusion, the enormous amount of cat-backs going into this, I actually think all this talk about an AI bubble is itself a bubble, meaning it is mostly being fueled by people who underappreciate, who do not really truly understand how fast capabilities are advancing, how fast we are seeing acceleration and the improvement in models, meaning models have been improving and I don't think anybody would deny their improving, but the pace of improvement is accelerating itself. And I actually think that, you know, at some point, all this talk about a bubble kind of has to pop because we're really seeing demonstrable evidence more and more and more of this value creation. I would love to see if we could share from our perspective to listeners what it's like to actually be inside building production grade AI systems and why we're so excited and feel like we're not inside a bubble. Okay. Let me turn to you first, you know, you wake up, this is what you do every day. Do you feel like AI is over hyped right now and we might be in a bubble from a capabilities perspective? From a capabilities perspective, I feel like there are significant progress going on and I don't see the sun of slowing down in terms of models capability, you know, based model core IOMs, you know, I also don't see any slow down of the harness around the core model. Like what we see in coding agents like clock code, there are innovations that's happening in both places, both the core model as well as the tool calls, the skills, the memory modules that people add around it. And Ryan really like, I think a previous point that you have made model times that, you know, even if the core model capability stops improving, the residual heat, I mean, the residual impact of the models are going to propagate for many years to come. I totally agree with that. And I think that is happening in terms of the harness that people are building around the model. But the core model, I'm not seeing it slowing down just, you know, not at all. Yeah. And I'm happy to dive, dive deeper if interested, but you know, I do want to dive deeper into that. Well, just to, you know, kind of get you on the record, are you feeling, I'd be honest here, do you feel like, yeah, maybe there's a little bit of over hype or no, actually for the average layman out there, they underappreciate just how fast things are moving. You know, I've typically leaned toward just being mixed to be honest. I mean, this week I was using clock code to automate something, and I was very impressed. I basically delegated a very complicated machine learning workflow over to clock code. And then came back with a very reasonable baseline, and that's something that normally would have taken, you know, even an expert practitioner days, if not weeks. And it's so incredibly hard to argue with the fact that you can do that now, and that we haven't even absorbed that capability into the industry yet fully, but I think a lot of the benefit is kind of isolated to folks that are really nimble, that can do a lot of, you know, they can adopt things really quickly, like developers. But I think that in the industry at large, there is like sort of a, like a human speed limit on adoption, where despite there being a whole lot of capability there that is just a hundred percent, like they could do, but like this imagine they could do anything that a perfect God-like oracle thing could do, there's still that human speed limit that we're running into now. But I think it's just the first time, maybe even in history, that we are such dead weight on the pace of innovation. Yeah, yeah. It's such a great point. It really is a good point. And so, all right. So I want to dig into this a little bit more, because I think this is a point that people miss a lot. And actually, you know, I really think that one of the reasons that we're so excited about AI is because you could argue that coding assistance have made the most progress of any sort of daily tool. Even beyond the chatbots, but I'm actually seeing, I'm actually seeing even in my normal day-to-day talks, if I talk to the marketing team at Kung Fu AI, they are seeing a shift. They are seeing a massive, massive improvement over the last few months. Like for example, Clawed 4.6, Opus 4.6 is even in their own perception dramatically better. It's not just in the coding agent side. Yeah. It's easy. I'm a real big believer that we've barely begun to absorb and understand the capabilities we have. We were talking before the show started that, you know, I will regularly have conversations with people that don't understand that spreadsheets can be analyzed and slides can be created. Or you could take a batch of 100 slide ducks and spreadsheets and just put them in a folder and ask the model to analyze them and find, you know, perform complex analysis and look for things for a fall short. So as much as I'm excited about the coding aspect of this, I think it goes beyond that. Oh yeah. Absolutely. 100%. Okay. All right. So let's segue like on a project basis, on a day-to-day basis, on a work basis, we're seeing a lot of improvement within AI. We have this special, we have this special perspective. Share with me examples that you think will help people really understand what it's like to be kind of in the eye of the hurricane. So I've been mainly using coding agents for coding and many other things because really, you know, a lot of people may be not realizing this when new to the field is, you know, coding agents, you know, they are good at generating code diffs, they can patch your code base. But they can do a lot more things like, you know, a good example is a micos example where, you know, you can let it operate your model learning workflow. And how long did your experiment around this, like, you know, just several hours or is more than that, I think? I mean, we have a beefy compute cluster that I was running on, so I feel like I should add that piece of info. But I think it was about four hours in total from the time, like, I had this collaborative discussion at the beginning to sort of do a lot of planning and task. The task was to make a Kaggle submission for Kaggle competition. Okay. And so, in case people don't know, Kaggle is this site where you can use machine learning to solve problems. So you were essentially trying to complete, trying to solve a problem, but using AI to do all the work. Yeah, I basically started with a conversation. I said, Hey, I want to solve this Kaggle submission. Here's some details about it. I've cloned the data locally. Here is how I roughly would approach it and some steps I would take along the way for like a lit review, exploratory data analysis, et cetera. So I was very prescriptive about the workflow that I wanted to use. But once I hit the button and hit go, yeah, it took about three and a half, four hours in total. And I went and submitted that submission. It's just a CSV file that it spit out. That is tested against a private, I guess the notebook I put together is tested against a private data set. So there's no way I could have known what that was and it was a competitive solution that was in the median. Yeah. And so meaning you was in this whole set. Yeah. Right. Exactly. And so that ran for a long, you said, four and a half hours? Not four hours, yeah. Yeah, about four hours. Okay. So let's get, you know, we're sitting here talking about some subjective experiences. I brought some data points on, on that topic, meter with METR, right, they have this data set where they're essentially doing task completion time horizon. That means a human task duration, so how long it would take human to solve a task and how frequently an AI can complete that task. And it's a little bit interesting because they do multiple, they'll do multiple attempts. So a model may get it right once, fail the second time, get it right the third time, et cetera. And they, they basis at a 50% so if the model can get it correct half the time, that's the score they're basically going on. That is now up to 14 and a half hours. Claude of this 4.6 can now complete tasks that take humans 14 and a half hours and it will succeed in half of those cases, which that's just astounding. And here's another set for you, Stanford, the 2025 AI Index Report reports an enormous one year jump on the software bench performance. It went from 4% in 2023 to 72% in 2024 and we all know that that's nothing compared to what we would see for the 25 numbers. So I mean, just enormous gains and we're also seeing it with our own work every day. And there's benchmarks like humans last, exam or last test or something like that and it's getting, and models are getting good numbers on it. So as long as there's a benchmark on it, it looks like model is going to beat it and saturate it. And I saw that open AI came out recently and they said that they are, they don't feel good about a lot of the software bench tests but they think they've been gamed and the models are actually performing better than that and it's difficult to demonstrate. So in my personal experience working on Climb Project, I've been using coding agents, several types of coding agents for the past year now. From in the beginning, using cursor, where it's a hybrid of editor autocomplete to more full autonomous, like just chatting with the code, the agent using English. And then just read the code and then approve it, approve individual edits to, now it's, I don't look at the code anymore even. I just ask and then the, like, code and codex seems to just understand my intent very well even if I just say a few, you know, just a few words. All right. So I mean, I know somebody listening to this is going to say, well, how do you know if it works? How do you know to trust the code? Oh, such a great question. They, they usually don't zero shot the solution, meaning they don't get to the correct solution in first shot. And I had a lot of pen points in it initially and here I have several ways to do with it. One is test driven. I will first, you know, ask it to generate some unit test, which I will review more carefully. And then, you know, if the solution doesn't even pass that, you know, it's, it's not good. And then the coding agent is able to autonomously, they can, they can self verify because they have the tools. They know the unit test. So they can write the code. They test it. Okay. It's, it's not passing. So it will keep iterating. Yeah. And then, you know, there's also the planning tool. You can first iterate on a plan and edit the plan that also increases the success rate. Yeah. And on that point, this is something again, where I think we have a special inside view, which is these, it's, it goes all the way back to that, that question about absorbing the technology capabilities as they exist right now already. Just knowing that if you put more time into planning and you iterate on coming up with a good plan, we'll tee the model up for success, not only for coding, but for, uh, report generation or analysis or whatever it may be, you, you, we're getting better at understanding how to use these tools and that's part of the return on the investment. I mean, something you all both kind of touched on in various ways is the fact that, you know, we can use AI to solve problems that AI created in the first place. Yeah. Like if they, you know, like the first solution to that right, you just ask it, hey, can you go fix it because you messed up X, Y and Z? From my experience, I think the hardest part about AI adoption for both, just like personal productivity use cases, coding workflows, whatever, is to just develop the mental reflex. To always come back and use AI whenever you'd normally would have tried to go like consult an external resource or whatever, if you just get over that one little reflex, I feel like it unlocks so much value and productivity. Yeah. AI is the, the, the answer to the AI problem, right? Yeah. Yeah. Just recursion and more. Very recursive. Yeah. Yeah. I want to share some pen points too. Like, you know, I'm, I'm sure a lot of engineers who use coding engines probably share the same pen is that it also generates a lot of slop. You know, just imagine how many, just, you know, see how many more poor requests are generated and, you know, those huge, that a lot of code, code changes, do you review them manually? And they're, they're like hundreds of PRs for a large ripple, you know, from, from code engines. Well, okay. And so, so on that, on that, on that point, you know, I, I would, we need data on dog feed. So you, you take those poor requests and you tell the model to, you tell the model to be more concise. Hey, co-pilot. Yeah. I mean, to review this, they are. To review this PR. This is, this is too verbose. You can, you can accomplish this much more succinctly. Yeah. Absolutely. Okay. A very practical outcome. So I was reviewing a PR recently that was created by a coding agent on one of our projects. And we did use co-pilot, like the built-in GitHub co-pilot agent to review the PR. And I genuinely was impressed. Like, it caught some things that I probably would have, if I went through with the fine tooth comb, things like handing versions of containers that were tagged as latest and, and stuff like that. It really is a wild time reliving them. Yeah. It is. Yeah. I, I found it most helpful in resolving the version conflicts in my Docker. I don't like all the other Python packages. Yeah. But verification is still a, you know, a pen, pen point that I feel like it's not soft. Like how? Like, okay. It automatically trains a model. How do you know the model's good? The benchmark number may look good, but then are you sure that the model during inference will give you a send results? Right. Right. Typically, you know, it's still a very manual, very out-hoged process now. And you know, we're increasingly in the world where the outputs are language-based. And so just even evaluating them is a challenge, right? You know, you get that recursive problem of LLMs as a judge, right? How do you judge the content, the quality of the model output? That is obviously a very big open problem. Okay. I want to pivot a little bit, guys. I don't want to ask you this. So like, I do want to talk about the capital expenditure stuff for a little bit, or let me just, let me just share a few thoughts. So one is, you know, having lived through the dot com bubble, it was a very different experience than this is right now. And we were basically almost at the end of the dot com bubble. It lasted. You could argue, you know, at the most, about five years. And I think, you know, from the IPO of Netscape to the bubble that popped in March of 2020. And the difference about that bubble versus this supposed bubble was the amount of capital that was going in during that time was all speculative. It was an enormous amount of cat-backs, but it was debt-driven, right? And so that's really, really different now. Yes, you see Google saying they're going to incrementally increase their op-x by 60 billion or whatever it is. But that is being funded through a free cash flow, which is fundamentally different than we're experiencing right now. I think that's a really important thing. And also the PE ratios are not really that out of whack. I mean, we had companies back in the dot com days who were trading at, you know, insane multiples, a hundred x multiples on phantom revenue. They weren't even profitable, right? These were, you know, multiples trading against supposed flow. As Nets has the raises R by open AI and Anthropic, they are already generating billions in revenue. Now, I'm not going to stand here and say that both those companies are going to succeed and that there's nothing to worry about. That's an enormous amount of money, but they continue to grow and they continue to put out models that are really, really differentiated fundamentally, though I do think open source is winning in a really big way because although that eventually filters down and in a funny way, most of the models that I'm using on daily basis are actually open source running on my laptop, which I wouldn't have dreamed of three years ago. So and then I've got just one more point and I want to, I want to get your, I'll stick on this. So one of the reasons I'm so, so, so excited about where we are and I wrote a blog post in this last year was this idea of reinforcement learning with verifiable domains and very briefly, again, for listeners, what that means is, you know, using reinforcement learning to train a model so that it can produce outputs that are verifiable. For example, you can generate a program, tell if the program works or you can generate a math proof and run it through lean or something like that to verify the accuracy of that math proof. I, I feel like this is behind a lot of the improvements right now, you know, programming languages are touring complete, meaning anything you can describe programmatically can be done. There's nothing outside, um, a touring complete language. You couple that with the emergent behavior and I think we've really, we've got line of sight over the next five years to a lot of gains. Okay. So all that was a big tip for this question. What if we're wrong? What if we're wrong? What if, what if we were at the top of the roller coaster right now and we're about to go over the edge? Let's just take for granted that yeah, there's a lot of games still be had from the AI that's been developed. It will take years to disseminate those capabilities. But what if we're at sort of plateau, we've hit the wall. I have a question for about you. What evidence would you need to see where you would say by 2030, um, if it doesn't happen, we've plateaued that we actually might be entering bubble territory. That's a great question. Um, yeah, I think I have bubble. I feel like, you know, um, as of right now, there is huge topics. So I feel like that could could be bubble, but it's really hard to say because I'm really imagining, um, you know, multiple trajectories of growth, the model capability, you know, it grows very quickly. The, um, business impact of the model, I think it's harder to measure. I think it's already, you know, a lot of substantial, um, values being created, not necessarily easy to measure using revenue because they're, you know, um, it can automate medical, um, medical analysis and, you know, say people's lives and, you know, that's hard to measure in, in revenue. Um, and then there's the, uh, capital spending like all the money that goes into data center and all the, all the GPUs, um, well, um, I think it really depends on, because, you know, capital spending, it, it can outgrow the capability growth or it can under, underestimated, right? So it really depends on, you know, uh, looking through that lens. And I think that's, I think you're right. I think that's probably the biggest concern would be, yeah, we're seeing gains, but, but the cat-backs is, is, um, even beyond that. So we're not going to see the return. Is it, how would you know? Is there some capability if, is it, is it, is it, is it the fact that maybe we might see a slow down in the acceleration of performance, um, for the core models capability and the Americas to, to see, uh, to, to hear both of your points to, um, well, if, if you're talking about models capability, um, let's look at the, kind of the three factors that, that determines it. There's the data, there's compute and algorithm data. Last year, I think we talked about the data wall. Did we hit the data wall because, you know, all the internet is being pulled down by overnight to, to train GPT and people were thinking, okay, did we hit a data wall? But then I think by now, I think people begin to realize that after your chat, GPT's released, uh, to many users and after coding agents, there's so much more data being generated, like all the thinking traces and also the user feedback to the thinking traces and coding that people applied to different verticals, you know, and also through different chats. Those data were not on internet before. Right. And now they're being kind of pulled out from people's brands and then gets fit into not, not to mention there, there's more and more evidence that synthetic data itself can be incredibly information. Oh, that's, yeah, yeah. That's a great point. Yeah. And come, come, compute. I don't say any sign off, you know, the hardware being, being a stock. Okay. So all right, Michael, I want to hear from you, but, but really quickly on the compute for it, because I think this is really an important point, which is one of the, one of the, uh, signs that we might be in a bubble is actually, we don't need that much power. We're, we're, we're spending too much money on, on energy or, and power generation. Or actually, we've, we've solved, um, algorithmically some, um, some problem in a way that massively reduces the amount of compute that we need. And so, Nvidia demand for, you know, frontier chips and things like that we reduce. Yeah. I personally think that that's not even remotely close to being the case, meaning that, um, take, for example, my reference earlier to the open models that are run locally. I'm using more CPU cycles on my laptop than ever before, right? Because I'm, I'm interacting with it just constantly. Ever warm. And those are very, very efficient models in the grand scheme of things. So I, I think that that also is just a complete red herring as well. Okay. So Michael, question back to you, what would you need to see so that you would change your opinion and say, actually, I think we're, we may be entering the hype, the realm of hype. You know, it's tough to say, I generally fall in the middle in terms of whether I think we are in a bubble or not. And I think that's because, um, I guess it's a little nuance, but I think that from a capability perspective, I've seen so much firsthand evidence that AI as an industry is 100% here to stay. I am like 10 out of 10. I would put all my money on the fact that that, like the, the concept of AI is almost like, it's going to be like a utility, like intelligence might be a utility in the future. Yeah, it'll be like electricity everywhere. It is, it is here to stay from an economic perspective. The thing that I think really is bubbly to, to put a silly word on it is that the distribution of that capital investment to a bunch of, you know, kind of like the runaway giants, like the frontier labs that I just don't see the moat in the long run. I don't see how, and I know that, you know, there's so much investment going on right now to put these data centers in place that, like, we're going to use that compute no matter what, like, I mean, ZZ and I were just banging down the doors this week at AWS trying to get access to GPUs. And the fact that we just can't for the life of us. And we have, like, this is for a client project. So we have the resources to fund this use case. And even after talking to people on the SageMaker team, etc. Like, we simply just can't get access to those. Yeah, that tells me that people behind the scenes, even outside the front, you're like, they're using these GPUs. But I just, I still don't see the moat. And I feel like a redistribution of investment. Like, they're going to be some big losers in the relatively near future because the other thing I constantly think about is that these big models right now have so much bloat in terms of memorization. When I think about a coding agent, those models are trained. They know the history of Rome and they know, like, they have a lot of memorization in place. And when I think about like the, the efficient size of that cognitive core that someone like Andre Carpathi thinks about. It's got to be substantially smaller than a, you know, 500 billion brand of a model or whatever the case is. And I love that fact because I think that the hope of people having personal, locally hosted frontier quality models is like, I think that's a realistic hope in the coming years. Yeah, yeah. Okay. And you bring up a really, I think, interesting point which is certain companies, the frontier labs in particular in these multi-hundred billion dollar valuations is, will they have differentiation? But I think that that is a very different question than, is there an AI bubble, right? That is very much like, hey, will the frontier labs will they be able to support this seemingly never-ending capital raise, right? Yeah. Yeah. Okay, Ron, I'm very curious to see your thoughts on, you know, what kind of events would you characterize as a tell-tale son of, hey, there's a bubble kind of like internet bubble? Yeah. I think, I think when capital starts going into companies that don't really have any differentiation, what we saw during the .com bubble was, all you had to do is say, you were something .com public and there was such a frenzy within the commercial markets to, to get a piece of the pie that companies like, you know, pets.com famously could raise, you know, billions of dollars and they were delivering dog food, right? Right. Right. I'm not saying that there will not be companies who add AI to their name. We see this all the time. Yeah. We see this all the time. The difference is the utility that was really, really apparent within the .com era was more diffuse, meaning all of a sudden it became possible for content to be distributed with essentially zero marginal cost, right? You could communicate at zero marginal cost and there were entire business realms that opened up because of that. But the value, the value was perceived as being implicit if you were just a .com company, like everything was going to instantly move online and they downplayed, I think, the, the physical realm. And that led to people overestimating how quickly it would transform things. Now all of the capital outlay around a high speed telecommunications. We were overbuilt for a while, but that's the foundation. That's the reason right now we can have all the, you know, the gigabit home networks and things like that. AI is different in a bunch of different, in several ways. One is the capital outlay is much, much, much more cash, free cash flow driven. If you look at the biggest companies andthropy can open AI accepted, they're not funding this growth through debt. And the diffusion of AI capabilities is dramatically faster. There is pretty much nobody I know, even my least technical friends that aren't using AI on a regular basis. And so the impact is really strong. Your point earlier is easy about, you know, how do you see this hitting the bottom line? I think that's a great question. I've had, I've had conversations with companies recently where they're saying things like everybody loves AI and they're using on a personal basis at work. We're seeing, you know, personal throughput meaningfully affected, but it's not showing up on the, on the, the PNL. Yeah. I think that is a completely reasonable perspective. And I think that's going to continue to, to happen. But that is not because it's not valuable. That is because, as we talked about earlier, it just takes a while for these technologies to be understood and disseminated. And you know, we'll look back, we're going to look back in 2030 and just laugh at how powerful we thought these models were because they're going to be much more powerful than that now. The models now are like idiots of violence and comparison. And we already look back with, I'd like chat GPT 3.5 and can you imagine having to use that now? Baby. Yeah, it was a baby. That was a few years ago. So I'm incredibly bullish. Okay, guys, that was a great conversation. I really appreciate your perspective as always. It's a lot of fun. Thank you. Thanks for having us. Thank you for listening to Hidden Layers. This series is hosted by Kung Fu AI, a management consulting and engineering firm focused exclusively on artificial intelligence. If you have any questions or thoughts about today's episode or if you know someone we should feature, please visit us at Kung Fu dot AI.