Battle for the AI Data Center: Deep Dive on the Semiconductor Supercycle

Stacey Raskin: I've been hearing the word super cycle, you know, for my entire career and like, maybe this is the first one I've actually seen.

Michael Marks: Now all of a sudden, hardware's cool again.

Stacey Raskin: Only thing we're hearing is that nobody has enough compute. You don't earn any money by training a model. You have to be able to use the model. So that is inference.

Michael Marks: When the PC first came out, there were 50 companies, and now there's three. And when the cell phone came out, there were 50 companies. Now there's. Hi everyone. This is the TechSurge deep tech podcast presented by Celesta Capital. Each episode we spotlight issues and voices at the intersection of emerging technologies, company building and venture investment. I'm Michael Marks, founding managing partner at Celesta. If you enjoy tech Surge, subscribe and leave us a review on your favorite podcast platform. Visit techsearchpodcast.com to sign up for our newsletter and find our video episodes on YouTube. There's a word that gets thrown around in the chip industry every few years. Super cycle. Most of the time it's oversold. Every so often it's real. Right now, the numbers are hard to argue with. The four biggest cloud companies are on track to spend around $600 billion this year, most of it on AI infrastructure. The semiconductor industry crossed $800 billion in revenue last year, and it's running toward $1.3 trillion. My guest is Stacey Raskin, who is a well known chip industry analyst for Bernstein. Unlike, um, most analysts on Wall street, he came up as an engineer, a PhD from MIT, who actually built chip manufacturing equipment before he ever wrote an analyst note. That shows up in how he reads the industry. Less speculation, more about the proven physics and capital flows. Today we get into whether the AI cycle is really different, where the bottlenecks are, and who captures the profit when the dust settles. So the question is now, now that AI is driving such big numbers, big dollar volumes, and you know, it's in the news all the time, which it never was before. Is something changed here or is this just another version of this cycle? Just a bigger version of it?

Stacey Raskin: Yeah, that's, that's the question, right? And it's funny, I've been hearing the word super cycle, you know, for my entire career. And like, maybe this is the first one I've actually seen. And what's really interesting, there's a couple of different types of semiconductor cycles, right?

Michael Marks: Right.

Stacey Raskin: There are, you know, supply cycles. Um, see these in memory. A lot like you, prices, supply gets tight, prices go up, you add a bunch of capacity. Capacity comes online. It turns out the supply, the demand you were building, the supply for was not real because customers, when they can't get the parts that they want, they tend to order even more. Right?

Michael Marks: Um, right.

Stacey Raskin: And so supply comes online, demand falls. Like uh, you gap out and it causes problems. So that's a supply cycle and they tend to last. You know those are your sort of your stereotypical like four year peak to trough kind of cycles, their inventory cycles, um, semiconductors at the back of the supply chain and small fluctuations in demand can kind of propagate backwards through the supply chain, have correspondingly larger impacts. And you know, if demand falls off tactically, the customers may buy fewer semis and bleed off their, their own balance sheets. And in that environment, semis under ship demand. And then it can, you can go the other way when they start to replenish and they over ship demand and they tend to show up as resets one way or the other and they tend to be order a few quarters. Right. Um, you can get product cycles. I'm, I don't know, I'll make it up. I'm a mobile semiconductor vendor making chips for smartphones and did I win or lose the socket in the next iPhone? And that can drive lots of upside, revenue or downside, those kinds of cycles. Um, this is like a demand cycle. It seems to be right.

Michael Marks: And there does seem to be.

Stacey Raskin: Yeah, right. And I've never seen, we get those sometimes. I've never seen one on this scale before. Um, the magnitude, um, and rapidity, like the speed at which this, this is actually like ripped is something that I think is somewhat unprecedented. Um and it's really interesting because everybody sort of focuses on the AI semiconductors and that's what's at the root of all the demand. Everybody wants to compute but we're reaching, it's starting to drag everything along with. So frankly you can look almost anything in semiconductors right now is working from a stock and from an earnings standpoint. And you've had this interesting move from category to category to category as one at a time these things have gone into um, constraint mode all of a sudden. Like we've utilized all the available supply for various different things as the demand for accelerators and GPUs has gone up. So you know we, we had, it went from accelerators to memory to semi cap tools to networking and optical to power semis and now, now even CPUs. Right. Everything is getting dragged along by this insatiable demand for AI compute. Um, and this is something I certainly have not seen anything on this scale in my career. Um, and so then, yeah, naturally the question is, what is, is this, is it different this time? Is there ever going to be a peak? Are we going to roll over? Like, I don't know. Um, all I can say right now is, is only thing we're hearing is that nobody has enough compute.

Michael Marks: Well, so why don't we just. Yeah, look, I mean, I, I hear you about that and obviously this is a huge thing. Why don't we, why don't we drill down on one aspect of it? Because it's, it's a complicated subject. Let's just talk about memory. Because, you know, memory, memory was the classic cyclical SEM, though.

Stacey Raskin: Yeah.

Michael Marks: In fact, it's so, so funny. You know, the guys that take turns on who is going to screw the other. So when, when there is not enough memory, the memory guys are charging up. And when there was too much memory, the other guys are driving the, you know, the customers driving the, the prices down. Now clearly, uh, you know, the memory guys are, uh, are in huge demand. They're sold out for a long time. So is it possible for this to go on for years or. I mean, one of the issues there is there's no new entrants, and so you've only got what you got in terms of capacity. So can this go on for, for years and years?

Stacey Raskin: Let's talk about memory. It's really interesting because this is probably the strongest memory cycle in history right now.

Michael Marks: Right? Right.

Stacey Raskin: Forget 18 months ago, we were in the midst of the worst memory cycle probably since the tech bubble. Right. And by the way, you mentioned there's fewer entrants. You go back, you know, 20 or 30 years, there were like 30 different memory guys.

Michael Marks: Right.

Stacey Raskin: There's like, depending on the space, there's, you know, three to six now, depending on the type of memory.

Michael Marks: Right.

Stacey Raskin: And you can go back to that, that nasty cycle that we're coming out of. Like, I mean, they were, it wasn't fun. Like, they were, they were losing money, but, but they were fine. You go back to the tech bubble days. I mean, we had bankruptcies in those, so we don't have those before. The, the memory industry is better from, from that standpoint. Um, and, and now clearly, like, I, we were seeing prices, you know, double every quarter and, um, demand is through the roof. And again, it really is being driven by AI and there's a few reasons for it. So I think it's important to separate the two primary types of memory. DRAM and nand, they have a little bit different dynamics. Um, those are the two biggest pieces. DRAM is kind of like what most people use for like the little uh, memory sticks that go into their PC and it runs, you know, the um, the system as you're working. And then NAND is like what's in your smartphone that, that you know, stores your photos say. And within DRAM there's some really interesting dynamics. So these AI chips use a lot of dram. They use a special type of dram, it's called HBM or high bandwidth.

Michael Marks: High bandwidth, Yep.

Stacey Raskin: Yeah. So what that is, it's a bunch of DRAM chips that are all stacked up on top of each other and then packaged together. And then each AI chip uses a whole bunch of these packages of, of memory on and a lot of it. If you were to think about the wafer or silicon area in an AI chip, it's probably 85% plus HBM M if you were to add up all of the chips, chip area itself. So it uses a lot of memory. And so not only is demand super strong, but there's another issue. They call it a trade ratio. What that means is to make say a gigabyte of high bandwidth memory because of all the stacking and everything you lose, you have lower yields. Basically like you, you uh, when you stack it, you don't, you don't get good chips out of everything you do. So right, the yields are worse. There's more logic dies that go into it as well and you need to leave space on the dies to put the connectors so you can stack the chips together. So because of that you need something like four times as met as much silicon area to make a gigabyte of high bandwidth memory as you do to make say uh, a gigabyte of standard dram. And so you're in a scenario where we could in theory be adding a bunch of like wafer capacity to make dram, but not actually adding that many bits because of this.

Michael Marks: Oh, that's interesting.

Stacey Raskin: Okay, yeah. And so for dram, you know, it, you know, as long as AI demand is growing like this can probably carry on. And clearly if AI demand rolls over then that would be a problem. But we're all screwed anyways if that, if that happens. So I don't, I don't worry too much about that scenario at this point and there's no signs of it yet.

Michael Marks: The question is, you know, you've been looking at this for a long time now and so do you see innovation, you know, easing the pressure here over time or is the demand just so overwhelming that even that can't make a difference?

Stacey Raskin: I mean innovation happens always. You know, it's really funny, there's been an overarching dynamic in semiconductors over, you know, the last six decades. It's called Moore's Law, which for many of your viewers have heard. And, but the idea of Moore's Law was every two years you were fitting like twice as many transistors in a given area of silicon and the cost of that area of silicon didn't really go up. So you were effectively every two years making m, you know, twice as many transistors for the same price or getting twice as much or getting the same number of, same number of transistors for half the cost. Right. You also got performance improvements. Um, the transistors got better performance and they use less power. So it was a wonderful thing. Um, Moore's Law started breaking down, you know, over, over 10 years ago and, but it didn't mean that you couldn't continue this technology. What it really meant is the cost leg of that three legged stool was going out the window. So we can still get performance and power improvements, but now you have to pay for it. Like the cost per transition is now going up and set it down, which is a new thing. And people were very worried that that would be the end of the industry and it really wasn't. It opened up a renaissance in the industry because what it actually meant was now if you want these improvements, Mr. Customer, you have to pay for them. Whereas historically the industry just gave it away for free. And I remember Henry Samuel, who's the CTO at Broadcom and he was the co founder at Classic Broadcom before Avago bought them and Classic Broadcom didn't. Analyst day, this is probably in 2012 and they were talking about this, right? And they were showing TSMC cost per transistors. I think they bottomed it like the 28 nanometer node, which is a long time ago. But his point was this is not a bad thing. The industry is about to become a rational industry for the first time in like 40 years. When Moore's Law ended, innovation did not stop. We just had to go into other things. And, and there was actually an economic like cushion, I think that actually helped to fuel this. And when you couldn't just like drive new, new functionality for free just by shrinking the transition, you did other things. You meant, you mentioned a few, right? But uh, they've gotten, they've gone to new transistor structures, right? They've Gone to. You talked about dicing the chips up, chiplet architectures.

Michael Marks: Right, right.

Stacey Raskin: I can, I can dice the functionality up into different types of silicon that are all sort of like, collectively, uh, optimized for what they're doing. And I can put those together in new and interesting ways. And that also lets you put much more compute on a single thing. Like, you know, to make a single chip, you can't make it much bigger than about 830 millimeters squared. There's what's called a radical limit. I can't physically make a single monolithic die bigger than that. But with some of these interesting packaging techniques now I can, I can put multiple dies together and build up a lot more. So you can put multiple compute dies. We're seeing that like Nvidia's Blackwell, um, GPUs is actually two GPUs stuck together. And then they got a bunch of these memory dies all stuck on there like next to them on what's called a silicon interposer. And you can build up much m. Much more silicon area and get more compute and do things. And again, it costs. Like these things are not cheap.

Michael Marks: Sure.

Stacey Raskin: But if, if you can give customers a reason to want to pay that cost, and clearly they do. I mean, Nvidia's got 75% gross margins,

Michael Marks: so there's so much to unpack here. If, if this begins to slow down in some way, what will be the sign of that? And.

Stacey Raskin: Yeah, yeah. So, I mean, the things that we watch, I mean clearly, you know, like the hyperscale Capex numbers, you know, by the. They like all reported on top of each other on Wednesday and still take Capex up. Um, but I mean that, that's one. And to be fair, by the time you see that go down, it's probably too late. Right, Right. Yeah, but that's, that's a very public, like, sort of thing we'd watch. Um, we're watching things for example, like, like, like some of the revenue numbers, um, at, at the labs. And by. This probably leads into a whole like, discussion which we'll probably get to on, on training these AI models versus using them. But for example, you can look at a company like, like an anthropic and they, they sort of like once in a while they'll, they'll talk about how, where the revenues are and I mean, they've gone vertical. Right. I think they, they recently said in April their annualized revenue run rate was like $30 billion. In January it was like 14 billion. So more than doubled in a Couple of months it was like 9 billion in December. It was, I can't remember was a billion or like whatever it was a year ago. I mean it's literally done this as people are now starting uh, to use but they're delivering people looking again go back to the hyperscalers and looking at their cloud revenues and how much are those growing and are they accelerating. Um, and then there's all of you know, all of the noise and data points and things that come out of Asia. People look at you know what are, what, what are the wafer volumes that are getting ordered, what are the co ops? Co ops is, is it's that packaging technology that's used to put the chips in the way memory together. How much COAS capacity is getting reserved by all these customers, all those kinds of things. Um, you know again you're looking at the different players in the supply chain and you know or how, how full are they? Are they sold out or they. Right, like right now everything is looking like this. It's, it's. The general consensus is there's not enough compute.

Michael Marks: At least, at least as I'm mostly interested in the sort of long term structural changes that are being created from all of this as opposed to the short term stock impacts and all this stuff. You just raised an issue on supply chain that I wanted to delve into a little bit because one of the things that's different and this time is how much vertical integration is going on at the hyperscalers. I mean that's something very interesting. They're designing their own chips, they're building their own data centers, they're setting up their own financing structures, they're controlling cloud access and so on. What do you think that impact is? I mean that's uni. The world's basically been vertically integrating and now we're going the other direction to be honest.

Stacey Raskin: The hyperscalers have been doing chips for a while. So for example Google has their own AI chips. It's they called a TPU stand for Tensor Processing Unit and you know they work with Broadcom and others to build these. They, they've been building these for 14 years. Like they're on their like 8th generation and getting ready for their 9th and like it, it's not new per se. The idea, the idea that they, that they're working Amazon does their uh, own chip is called Trainium. I can't remember how many years it's been five or six years. Um, Amazon in particular on the CPU side has been, they have a C, A ah server CPU that, that's their own design, it's called Graviton. They've been doing it for, I don't know, probably six, seven, eight years, maybe longer. So the idea of the, at least the hyperscalers vertically integrating the semiconductors is not necessarily new. Um, again, you know the, the demand profile that they're building for is certainly much bigger than it used to be. But the idea that they need to vertically integrate into that is not new. Um, not even the hypers that take, take it up like an Apple. So Apple does a tremendous amount of their own silicon. You know they do their own, they move migrated their entire PC platforms off of, off of intel and what's called the x86, uh, compute architecture to their own custom arm. Arm, what's called arms, the company that does the licensing, uh, ARM based architecture. Um, they're starting to do their own uh, cell phone modems, radios that Qualcomm currently supplies and that's going in house at Apple. Um, they do some of their own connectivity like Bluetooth and wi Fi and other things. And you know, so Apple hat, for example has a very strong internal. So they've been doing that for like, well, well over a decade.

Michael Marks: Okay, but let me set, but let me separate those if I might because Apple is a hardware company whereas you know Meta and, and Microsoft and data

Stacey Raskin: centers, you know, even on the data center side, you know, like for, for Google for example, like they're not really necessarily like buying their service, you know, from Dell. Like they, they have a white box model so they sort of spec out exactly what they want and then they go, you know, like these guys, you know, ODMs in Taiwan and they'll build it to their, to their specs. They'll use like off the shelf silicon, you know, from Broadcom or whoever. But I mean they've got pretty strong hardware chops too. And I don't think that's fair enough, you know.

Michael Marks: Fair enough. Well, do you think that, that this vertical integration is a, is being done to, to avoid bottlenecks or is it being done to create more margin or is it more protection against competition?

Stacey Raskin: I think it's all of the above. So I mean it's bottlenecks because you don't want to be, you know, some of it maybe, uh, although even they're like they're not actually physically building the stuff, right? So you can still get bottlenecks. Some of it is performance and spec like you can because they have, think about it, these hyperscalers have very large stable like internally developed workloads, so they know exactly what they want and they can optimize for it. And so if it's your own sort of custom designs, you can optimize everything very, very closely and get better performance on that basis. That's good. There's probably an aspect of competitive threats. I mean, you don't necessarily want to be fully dependent on a single sole supplier. And so, like, on the AI side, I mean, these guys buy a lot of stuff from an Nvidia, but I mean, they do their own work as well, right? Sure.

Michael Marks: Do you see other bottlenecks developing in ways that we don't hear about very much? For example, one of our companies was Credo, which went public a few years ago at 11, and now it's at 175. They're buying another one of our companies, D Photonics. These are things, you know, most people who are following stock markets, if you will, don't know about optical connectors or, you know, we have a company that uses. That's making silicon timing chips. And, you know, there's all these little things that are happening that are. That are interesting from an innovation standpoint, but they could also cause bottlenecks because these are smaller companies.

Stacey Raskin: Sure, sure. Yeah. I mean, they're. I mean, they're all sold out, right? Yeah. So the bottlenecks have been interesting because people have been playing the bottlenecks. I know you want to talk too much about stock prices, but just in general, you can sort of look at, like, the stock prices of, say, like, the compute names, and they kind of stagnated for. For a bit. And yet you can look at, like, the memory and the semi cap and the optical, like these. These bottlenecks, which have all ripped. And so people have been really playing the. The constraints. Right. Um, yeah. It's funny too, because, like, in some sense, one of them is wrong. Right. They're not. Those. Those two trends are not really consistent with each other. So it probably does have to normalize at some point one way or the other. But. Yeah, but there's been a lot of bottlenecks, and I found it really interesting how you sort of bounced from one to the next to the next. I feel like it's almost as the AI demand has. Has taken off. All of these guys have, you know, whatever it is, X amount of capacity that they can bring into the market, and we've just been hitting those levels one at a time for, like, the different areas at different points in it. So like I said, you know, it was memory and it was optical. And it was semi cap and it was power semis and CPUs now. And I think people are starting to look deeper into the supply chain of substrates. And you know.

Michael Marks: Right.

Stacey Raskin: Like a lot of the companies now complaining about like their own input costs going up because, like, even the raw materials are now starting to get tight.

Michael Marks: Their own what Costs going up?

Stacey Raskin: The raw materials, you know, like, okay, substrates and T glass and like all that. Intel had talked about some of this. Intel reported earnings like last week. And one of the things they talked about was like, with their input costs are going, so they're trying to raise price on some of the stuff. By the way, as an aside, you know, the industry collectively has had pretty good pricing power. So during COVID we had a massively inflationary environment.

Michael Marks: Right.

Stacey Raskin: And all of these guys had their input costs going up. Broadly, the industry was able to pass those costs along. And they did it at their margin structure. So it's like, you know, I've got a 60% gross margin and, um, my cost went up 100 bucks and I'm passing along 160. And they all broadly did it. And I didn't have a single company of mine complain about margin compression because of input costs going up.

Michael Marks: Yeah, yeah.

Stacey Raskin: Interestingly enough, if you look at the industry, you compare 2019, which is the first year pre Covid, versus say 2024, when everything finally normalized. Right. Units, um, were roughly for the total industry, collectively were roughly flat 2019 to 2024. ASPs collectively 50% higher.

Michael Marks: Well, we get into this a little bit, might as well do it now because you're really talking about oligopoly pricing. So we do have a situation here where there aren't very many suppliers. And when a lot of demand and not very many suppliers, you can raise prices regardless of the environment.

Stacey Raskin: Uh, not always, but I mean, prices stronger. And it's really coming to memory. So, I mean, you can look, I think in February, industry revenues were. It was something crazy. It was up something like 80% year over year. And it was almost all memory pricing, to be fair.

Michael Marks: Right.

Stacey Raskin: Memory pricing is. I mean, it's gone ballistic. Um, and it's sort of interesting for a long time, people coming out of COVID we're talking about a semiconductor industry that could be, you know, a trillion dollars in 2030. Right. Uh, we did roughly a little under 800, $800 billion last year. This year we are already on a run rate to exceed a trillion dollars. I think we did over $100 billion in February. Now, to be fair, A lot of that is memory pricing and Nvidia gross profit dollars.

Michael Marks: Right, Fair enough.

Stacey Raskin: Um, but even so um, it's Maybe we're doing 2 trillion in 2030 now, I don't know.

Michael Marks: So do you see any, any other new entrants into memory? For example any of the hyperscalers?

Stacey Raskin: Because you know the Chinese are probably the biggest entrance. They're already there. There's two primary Chinese players there, CXMT who does DRAM and YMTC who does um nand now Flash and um, they're a little constrained. You know they've been on. The US has sort of put them on sort of some regulatory blacklists and so it's been harder for them to do business on, on a, on a, on a global basis. And like, like for example the semi cap guy, at least the US players who sell the semiconductor manufacturing equipment have not been allowed to sell tools to them.

Michael Marks: Right.

Stacey Raskin: Um, they're capable though especially YMTC on the NAM side is pretty capable, you know.

Michael Marks: So do you think?

Stacey Raskin: I, I don't know like it's, it's, it's tough, you know, I don't know.

Michael Marks: The hyperscaler giants are spending somewhere north of $600 billion this year on this build out total AI revenue. Everyone's combined is still well short of that. Spending on AI infrastructure reached about 4.4% of US GDP last year close to the peak of the dot com bubble. Optimists think that demand is racing to fill the gap and they can point to real fast growing revenue. Skeptics think a lot of this capacity is being built on faith. The thing that will decide this isn't the models, it will be inference. People actually using the models every day and paying for it. These cycles are interesting to me. One of the things that's happening, we've got this whole move to inference which I'm sure you know plenty about. Talk about all the time. Let's talk about that for a little bit because one of the things that's happening is that's a, a bit of reordering the deck chairs if you will because now there's sort of a new game in town. Um, I'm going to come back to the specific players but, but just any comments about what's happening in the industry? Because inference is taking over you.

Stacey Raskin: You bet. So a lot of the spending has been on training and basically I need to, to, to determine the, the potentially trillions of parameters that need to be set in these big models for them to be able to do stuff right. Very Compute intensive requires a lot of, a lot of money and they buy a lot of GPUs and other stuff to do this.

Michael Marks: Right.

Stacey Raskin: I um, usually get the question a lot. Well, when do you think the spending, do you think the spending will ever like pivot from being training dominated to inference dominated? My response has always been will it better because you have a problem? Because you know, training is important, you

Michael Marks: need to do it.

Stacey Raskin: But like you don't earn any money by training a model. Like Nvidia earns money by you training a model, but you buying the chips does it. You have to be able to use the model. So that, that is inference.

Michael Marks: Good point.

Stacey Raskin: And again I think we're starting to see it. Um, so it, and it's beyond just the, you know, the chatbot stuff and, or you're making slot videos or whatever. Um, people start studying, very excited, what's called agentic inference. So I've actually got the model that can actually go out and really perform real world tasks for me. And um, again coding is probably the application where we're seeing the biggest um, uh, piece of this now. And again I mentioned anthropics revenues, but clearly a lot of that, most of that is agentic coding where you've got these agents that are actually helping you or doing code for you and clearly they've got something that people are willing to pay for.

Michael Marks: Right. I do hear that people are getting a bit shocked when they get their bills though.

Stacey Raskin: Yeah. It's funny too because a lot of times we're laying off employees to spend on AI and I said well wait a minute, we're spending more on tokens than we were on the employees at those tokens are supposed to be replacing. Yeah. Now I guess if those tokens are more or more productive, maybe it's, maybe it's okay. Right. Um, but, yes, but we, I think we are starting to see that still early. We're clearly starting to see it. And frankly it's some of the things that you're seeing concerned, like for example the CPUs, those are getting constrained in part because like a lot of this inference of it doesn't. It uses a ton of GPUs, but it uses a ton of other stuff too. Right. And the fact that we are seeing those kinds of constraints now is, is another real indicator that, that we are starting to see a pivot toward inference. And again we can still I think have justifiable debates on the return. And, and what do the economics look like? I think the industry is still working through that. And it's sort of hard to say because we're in a ramp phase. In a ramp phase, like you're always going to have high capex. Right. And Right. It looks like it's burning cash and everything. So you have to sort of normalize out what is it going to look like once the investment's made and how much profit can you, can you derive. But, but the more the revenues are growing, I find that very encouraging on the inference front.

Michael Marks: So you have these new inferences now, I think. Yeah, yeah, yeah. Well, one of the things I want to talk about is one of the implications of that is you have these, you know, inference, uh, focused startups like groq and Cerebra, Sabinova 10 storage, you got this new category of companies that are not in the public knowledge domain as well as Nvidia and Broadcom, these others, how do you see that playing? I see them all get bought up by the same players. I mean.

Stacey Raskin: Yeah, I mean it's interesting because. So for example, like Nvidia just bought Grok. Why did they do that? And you know, they just had invited their, their GGTC event, the GPU Technology Conference. They do it every March in the Bay Area.

Michael Marks: Right.

Stacey Raskin: And Jensen was. They just bought rock. So they were talking about it and I finally understand like what was going on. And one of the takeaways I took from there was I kind of had this just sort of dumb, simple view that like a token is a token is a token. And by the way, when I say token, a token is sort of like the chunk of data or information that these, that these models are based on, either that they're taking in or they're spitting out and we can go think about it as like a word or like a piece of information like generated or used by the model anyways.

Michael Marks: Yep.

Stacey Raskin: And when you're renting, when you're buying capacity, many cases you're paying like by per million tokens or whatever.

Michael Marks: Right.

Stacey Raskin: So I'd always just thought about, okay, you know, a, uh, token is a token is a token. I'm renting out X number of tokens. And what Jensen said, which kind of makes a lot of, like stupidly makes a lot of sense when you think about it, is they're not all the same. And some tokens can be monetized at a far higher rate than others. And in particular the tokens that require really low latency, like really fast responses. Jensen's view is that those, if you're say a NEO cloud and you're renting this capacity that you can rent out, that kind of capacity of much better economics. That's why he bought Grok, and that's what GROK was for. And some of these inference focused, um, startups and others you mentioned, that's kind of the idea they're focused on. So it's not for everything but for that certain subset of tasks, and I'm grossly simplifying here, that certain subset of text that require like really fast responses, really low latency, um, that can work really well. That's not a thing that like a GPU is necessarily like the best suited for. So that's why he bought, he bought

Michael Marks: Grok and yeah, that's what I understand. Aren't the rest of these guys.

Stacey Raskin: Jensen's admitting that GPUs aren't the best for everything? But to be fair, Jensen's also big and, you know, big enough and not full of himself enough to he can admit when, when he needs to fill a hole. Right.

Michael Marks: Well, it seems to me like all these guys are gonna, that when I say all these guys, you know, you've got a very small group of players and now you have this, this pivot to inference which we're talking about. And so Nvidia, but Grok, but these other companies seem like they're all going to get bought up as well.

Stacey Raskin: Maybe. I'm not going to speculate on potential M and A here.

Michael Marks: Yeah, of course.

Stacey Raskin: But I mean, yeah, you know, they've got assets that I think at least the Grok acquisition that it validates.

Michael Marks: Right. Well, one of the things that I think about, you know, having been around for a long time just like you, is, you know, when, when this, when the PC first came out there were 50 companies and now there's three. And when the cell phone came out there were 50 companies, now there's three. And we're going through a lot of that right now in the AI revolution. That's why I think there's what I refer to as vertical integration. So I'm not, I'm not trying to speculate on any particular company, just looking at the how changing from a structure standpoint.

Stacey Raskin: Why don't we talk for. Right.

Michael Marks: One of the things that doesn't get talked about very much, and I know you're a follower, is the semiconductor equipment side.

Stacey Raskin: Yes.

Michael Marks: If you look at their stocks, they've all gone up at nowhere near the rate of these other players. So what's going on there?

Stacey Raskin: They've gone up a lot. I mean, uh, just to pick on one Lamb, you know, who makes, they make action Deposition. I mean that stock at the beginning of last year was $70 and it's, I can't remember what is now 280 or something. It's tripled or quadrupled.

Michael Marks: Applied Materials things doubled this year. SML has maybe doubled.

Stacey Raskin: They haven't gone up 10x. Like, like the memories. And you got to remember, like I said that I'm gonna make up the numbers I got. I don't know what it is but I mean Micron, you know, Sandisk or something, you know, they're, they're guiding to a higher eps, you know, than, than the stock price was, you know, 18 months ago or whatever.

Michael Marks: Right, right.

Stacey Raskin: You're not seeing those kinds of revisions.

Michael Marks: Right. But what are the impacts on, on equipment? Because it's a longer cycle and yeah,

Stacey Raskin: like to be fair, like in memory, you know, it's pre, it tends to be price driven. So I mean again, I'll make up the numbers but if the ASPs are going up 10x, you know, the stock semi cap ASPs are not going up 10x. Right. And frankly semicap's, uh, interesting because like this is going to probably be a pretty strong year for, for what? For what's called people. People sometimes they call this WFE Wafer Fabrication Equipment. So this should be a pretty strong year for wfe. But as strong as it is, it's a constrained year because if I'm building, if I'm selling semiconductor manufacturing equipment, I need somewhere to put it. I need what's called a fab. The factories are called fabs. I need a clean room, or sometimes they call it a shell. I need a clean room, uh, a factory to put the equipment in. They don't have the factory. So I have to build the factories first. And so as strong as this year, that is a constrained year. So they're building the clean rooms. Those will start to come online and be available for, to accept shipments like next year. So that's sort of a constraint and, and frankly, you know, people probably, somebody will probably talk about AI bubbles or not. Uh, but I don't think this is a bubble. We're not anywhere near crazy enough to be bubble yet. We can talk about what, what we might need to see. But in some sense this is sort of a mitigating factor. Like there's, there's in some sense a hard physical limit on how much the industry can expand. I bet if like the clean rooms were unlimited, we'd be selling a ton more equipment this year. And whether or not it would actually be shipping too much. I don't know. Right. But we'd be shipping a ton more. So there's. There's this sort of natural constraint on how quickly things can ramp because we just don't have the semiconductor capacity to do it. And so it will, it will take time to bring that online.

Michael Marks: So following up that you. I would, uh, surmise that what do you think is there will be a long upcycle for the semiconductor equipment guys just because they're going to be filling a backlog of demand for many, many years, probably.

Stacey Raskin: Unless. Unless AI demand rolls over again, in which case we're screwed anyways. So who cares, right? I mean, that's.

Michael Marks: Are there any new entrants worth watching in the semiconductor space? Equipment space?

Stacey Raskin: There are those. So there are, there are some. I mean, there, there are, you know, there's the big five. There's this ASML, Applied Materials, LAM Research, Tokyo Electron and KLAC.

Michael Marks: Yep.

Stacey Raskin: And those five collectively are all roughly 70% plus of total WP spend. There's a long tail of more niche or specialized guys. You got like ON two and Nova and I don't know, there's smaller like subsystem guys, mks and I Core and they make like subsystems that go on the tools. Um, I think where people look for, for others is, um, the Chinese. And there are some Chinese companies that make the stuff who are actually pretty. They're pretty good at what they do. They're not as good as the US Guys. They can't do everything, but they've been taking some share in m part because of the regulatory environment. The US players have had limits placed on them in terms of what they are allowed to sell into China. Because of that, the Chinese players, I mean, they're probably taking more share than they would ordinarily deserve to take because they have no choice.

Michael Marks: That's interesting.

Stacey Raskin: It's been manageable. It's been fine. Demand overall, even for the US Guys, even in China, has gone up rather than down.

Michael Marks: Uh, yeah. What's funny, because of all the geopolitical stuff and, and you forget that, that there's a real source of, you know, some of these bottlenecks we talked about earlier can be alleviated by the Chinese. That. That's a good point.

Stacey Raskin: Over time. And look, you know, there's. That's a whole other debate. Is it a good idea to limit what you're selling into China or not? Because then you encourage them to be creative in all senses of that. We're seeing that on the AI chip side right now because they're not allowed to sell AI chips into China. And so China has local players that are making AI, uh, chips that are not nearly as good, but they've got a lot of power at least. So they can brute force these big clusters, they don't care about energy efficiency. And in 10 years, what does it look like? I don't know.

Michael Marks: So since you know so much, I'd like to ask just a couple of things from a company standpoint. I think there's nothing left to be said about Nvidia, but it would be interesting for our listeners if you could talk about how Broadcom has taken advantage of this because Broadcom is not as well known as Nvidia and Intel in terms of their chips, but they do some things incredibly well. Could you talk about that for a minute?

Stacey Raskin: Amazingly well. Yeah, I like, I like Broadcom. Um, it's got great leadership, um, and their execution is top notch. And so what is brought to. So, so before all this started, Broadcom was sort of like an agglomeration of lots of things. But the company had grown through acquisitions over the last 10 or 15 years. Um, and actually the reason it's called Broadcom, the uh, ticker is Avgo, which

Michael Marks: is, I know Avago was called Avago

Stacey Raskin: and Avago actually bought what I would call today class Broadcom and kept the ticker but, but took the name.

Michael Marks: Right.

Stacey Raskin: Um, anyway, so Classic Broadcom did um, they did wireless stuff. They did like RF filters and the Bluetooth and the WI fi stuff that's in the iPhone.

Michael Marks: Yep.

Stacey Raskin: Um, they did uh, things like uh, broadband set top boxes and cable modems and things like that. Um, they did storage, storage chips and things. And then their most important business was networking. And they had actually uh, they'd done custom networking chips. And then when Evago bought Classic Broadcom they got what was called a big, what they call merchant silicon based switching and routing. So that was Classic Broadcom.

Michael Marks: Right.

Stacey Raskin: Then they started getting into software and they bought CA Technologies.

Michael Marks: Yep.

Stacey Raskin: They bought which, which did make does, didn't does mainframe software. They did um, and they bought Symantec Secure enterprise security business. And then they were recently bought VMware that does virtualization. And so before the big air on air and started they were probably roughly 60% semis, 40% software, really high margins. Um, and Hawk Hock Tan, the CEO and sort of always talk about semis are a mature industry and we're probably growing mid single digits. Uh, but we generate a lot of cash and the multiple Was, was reasonably low. I always thought the mold was too low, but it was pretty low. Now as part of this. So they've, they've always done custom chips, custom networking chips and custom what, what he calls compute offload today, the AI stuff. But it was compute offload, compute accelerators. And I mentioned earlier, Google's been doing this for like 14 plus years working with Broadcom.

Michael Marks: Right.

Stacey Raskin: And it was not a big business. I can't even. It was a billion dollars a year, whatever it was. Right, fine. Um, and then the AI thing started. Right. So Nvidia, ah, you know, chat GPT showed up in November 22nd. Nvidia sort of had like the print heard around the world in May of 23. That was when their big revenue ramp started. Um, and then Google and the others started to get more excited. And I'll be honest, I remember my own view. Hawk started and Broadcom started talking about it around kind of like second half of 2023. And it was funny they were talking about it, but you could sort of tell he didn't exact. He wasn't sure if he believed it or not, I think.

Michael Marks: Right.

Stacey Raskin: But they sort of said we're starting to see this, you know, and it's networking and it's. By the way, we even talk about network networking is as important as the compute in all this. But we're seeing upside networking and compute.

Michael Marks: And.

Stacey Raskin: And then it just exploded. Right. Um, and I mean he, he. Their last, or just to give you some context, their last earnings call, they said next year they think that they could do $100 billion in AI revenues next year. In fact, they'll, they'll probably do more, more than that.

Michael Marks: Right.

Stacey Raskin: Um, just to give you some scale of some, some idea of, of, of where the revenue scale was. I. This is a company that used to do, you know, tens of billions of dollars of total revenue. They'll do 100 billion-plus in just a revenue next year. Um, and it's really taken off because within the AI space you've clearly had Nvidia who's been dominant, but there are large players, um, who have in many cases were already doing their own chips who number one want to customize their chips for their own specific AI workloads to get higher efficiency and higher what's called total cost of ownership TCO. And then there's always the competitive thing like Nvidia's. You know, they've got 75 gross margins and they charge a lot of money and at a minimum you'd love to have something in Your pocket when you're sitting across the table from Jensen, like negotiating on next year's contract.

Michael Marks: Right.

Stacey Raskin: So yeah, um, and, and I, I, I like, I like both the GPU and, and the, the ac, what they call the ASIC stands for Application Specific Integrated Circuit Custom Chip is what it means.

Michael Marks: Right.

Stacey Raskin: I'm bullish on the GPU and the ASIC side. Um, I figure ASICS probably will take share. You're coming from a lower base, but they're not universally used.

Michael Marks: Ah.

Stacey Raskin: ASICS tend, their custom chips tend to be used for what I would call large, stable, internally developed workloads. So large because it's expensive. So you need to have a lot of workload volume to amortize that that over in order to make it cost effective stable. Because like that's the trade off. Like I'm willing to stipulate that the cost of ownership for an ASIC should be lower for the workloads that it is designed for. Because otherwise why are you bothering? But it's not like it's magic, like it's a trade off. There's no free lunch. Like if your model structures change, if your workload change, you need to design a new chip whereas the GPU is programmable, so you gain flexibility. Right. But if your workloads are stable and you can design the chip specifically for that, you should be able to get better, better economics and then internally develop because like you want to design it for, for your own specific needs. So that's probably not, you know, 80% of the market. I mean today ASICS are probably, if you were to add it up, it's probably mid teens of the, of the revenue shipments, it's probably higher on a unit basis, probably mid teens on a revenue basis. Could that number go, could it be, I don't know, 25% or 30% of a much bigger pie?

Michael Marks: Yeah, maybe so getting to be a pretty big number. Yeah, that's also reasonable.

Stacey Raskin: But again I don't, and it's a big controversy by the way, the whole TPU versus GPU and who's winning or losing. And my view has been it's the wrong question, the right question. I keep coming back to it. The right question is, is the opportunity still friend of is still bigger, is it not? Because if it's big, I think they both thrive. If it's not, they're both screwed.

Michael Marks: Well, that's a perfectly good segue to the other company I wanted to just touch on briefly, which is intel. And you know, and in full disclosure, I think, you know That I started this firm with the Bhutan. We started it together. And I'm a big fan of his. I know, I've seen your CNBC stuff. Maybe it's ahead of itself and price and stuff. But let's talk about a couple. Let's talk about the foundry part of the business, because, I mean, there's a lot of argument Foundry doesn't belong, blah, blah, blah. But, you know, there's real issues in foundry capacity here in the world. Geopolitics and TSMC and being under the, you know, the watchdog, all that kind of stuff. So how do you feel about the foundry business?

Stacey Raskin: So, look, I think. So let me step back. So this is sort of Pat Gelsinger's maybe, right? His whole idea, and I don't think the strategy was. Was wrong. I think Foundry is important and both from a national security issue as well as just say we need it. I think Pat's execution on it was not good. Um, you know, he was. I mean, look, I don't know who comes into a turnaround story and acts like Pollyanna, which. Which he did, right? He started, he hired 21,000 people and then he had to fire them all. And I mean, it was. I mean, Lipo is doing a better thing, which is you come in and you under promise instead of over promise, and you got the cost structure in place and it's like, look, I'm so happy to be back, but, like, you're gonna have to be patient. Like, it's gonna be a slog. Right? That was the right response versus, like, Pat coming in and saying everything is perfect forever. Right?

Michael Marks: Yeah.

Stacey Raskin: Um, so I. And by the way, I like Lipu. I've known Lipo a long time. I like lipo a lot. He's doing the right thing. He's not a magician, but he's very good. He's probably what they need. Um, he's technical. He knows how to run a foundry. Right. I mean, basically founded SMIC and he was on the board there for 20 years and whatever. And, you know, he's. He's involved with tons of startups who are all TSMC customers. So he knows what customers want and he knows everybody in the industry and everybody. He's a relationship builder. So he can pick up the phone, he can call cc, he can call Jensen, like. Like whatever, right? Is what they need. Um, it's still a slog, though. And to his credit, he hasn't denied that, right?

Michael Marks: No, he certainly hasn't, no. But there aren't very many foundries in the World, it's another one of these oligopolies. And so there seems to be a

Stacey Raskin: need, I think with intel, there's no question that, that we need the foundry. That the question is, can they deliver on it or not? I think that's still tbd.

Michael Marks: Right.

Stacey Raskin: And, and so, so what do you think about.

Michael Marks: What do you think about their shareholder base now? They have the US Government as a shareholder, they got Nvidia as a shareholder, they got SoftBank as, uh, is this good for them as a public company?

Stacey Raskin: Does it really been good for the stock?

Michael Marks: I will say, but this isn't a show about stock performance. Near term stock performance.

Stacey Raskin: It's funny though. I mean, look, I'll be honest. I made my career being negative on intel and it was, it was the gift that keeps on giving for like 15 years. 15 years. And I mean, clearly, like, the market has another opinion now. Um, and, and, and that's fine. Like I said, we need them. And at a minimum, I would say because of these investments, you ask, is it good or bad? Like their balance sheets in a much better place. Yeah, that was a real question for a long time. Like, are they going to be around or not? Right. So I don't think that's a concern anymore and that that is valuable.

Michael Marks: Right.

Stacey Raskin: I think the government stake it. Like, I don't like it, you know, but at the same time, did it give other folks a vote of confidence? Like, maybe. Right. Um, I mean, look, you got Trump like tweeting out like every other day, like, how great his investment intel is. Like that, that's a, that helps. Right. It gives other people. Um, but I think the fact that, I mean, I mean, look, if they really are going to pursue this strategy, they need a lot of money.

Michael Marks: Yeah.

Stacey Raskin: And it was really questionable, like, for a long time.

Michael Marks: Right.

Stacey Raskin: Whether or not they were going to be viable. And there was, there was talk for a while that they were going to have to get carved up for parts. Right. I think that's off the table now. Um, and I think from that standpoint that the investments from the government, the others are valuable and clearly are positive. Yes.

Michael Marks: Well, you know, you touched on something that's near and dear to my heart, which is balance sheets, which nobody talks about. And you're exactly right. With intel, they were in a world of hurt from the amount of debt that they had coming due, and people weren't talking about that. Uh, but you're right, one thing about having these investors is that now they have a much stronger balance sheet. And that takes some Risk off the.

Stacey Raskin: The table.

Michael Marks: I think we can. I think we can agree. Everybody, more or less, is pulling for intel to, to survive here and to be a strong company.

Stacey Raskin: I mean, like, and. And to their credit, I mean, let's talk about the good things, right? So, um, their new products, their 18A product, Panther Lake.

Michael Marks: Yeah.

Stacey Raskin: Looks like a great product. I wish they could make more of them. I mean, they're. I mean, you know, they talked about, like, the yields are better than they thought, but I mean, clearly. Do you look at the margin? The yields are still not.

Michael Marks: Not good. Right.

Stacey Raskin: But the products they're making with them are good.

Michael Marks: Right.

Stacey Raskin: So the process itself, if they can get it to yield is, Is. Is good.

Michael Marks: Right.

Stacey Raskin: Um, so that's, That's a, uh, that's a positive. Um, and frankly, on, on the server side, I mean, their products are. And servers are not good. And Lip, who's admitted this, and he said, you know, we won't really be competitive until we launch Coral Lake, which, uh, Coral. Coral Rapids, which is, I don't know, 20, 28 or whatever. But, uh, the environment is good, right? And look, I mean, you take Lucky over good any day. Like, it's. It's fine, right? I mean, take it when it, when it. When it happens. The cpu, server. CPU demand is really strong right now. It is so strong that last quarter they had some margin upside because they were selling like, uh, previously written off, like garbage that was like, lying around, like in the corner of the warehouse someplace. Demand is so that they've written off to zero.

Michael Marks: Right?

Stacey Raskin: The demand is so strong, the customers are like, we don't care. We'll take it. Please sell it to us. So that may, that may help whether. Whether it's luck or not, it doesn't matter, right? If the server demand continues that, like, that, that may help a lot.

Michael Marks: Yeah, I agree. Look, this is fascinating. Before we get wrapped up here, as we're wrapping up, I want to talk a little bit about the future. A couple of different questions I like to ask you. One is, so everybody is all up into the right, blah, blah, blah. What are people missing anything? Is there, I mean, are there risks here that people aren't paying attention to?

Stacey Raskin: Yeah, I mean, I mean, look, so I think the question of returns are still, uh, a valid debate because you can make the simplistic argument, well, of course, demand is off the chart. Like it's free, right? So how are they going to monetize this? And again, I am very encouraged by seeing some of these revenues, some of these revenue Footprints go up a lot. But I mean, I think that's still an open debate. How does this stuff ultimately get monetized? I think there's a debate, you know, how many foundational model companies are actually going to be around. You know, we've got a bunch of them, you know, and all of them are not.

Michael Marks: And, and we're not going to eat all of them. This is now a bad boy business.

Stacey Raskin: And if some of them go away, just do the other ones that are left absorb the capacity or, or not. Like, I think those are, those are debates. Um, you know, where is the power come from? So that's something, you know, if Jensen said at one point we were going to be spending three or four trillion dollars a year on infrastructure and it sounded crazy at the time, although I'd say that we're probably getting close to a trillion now. So 3 trillion off of a trillion doesn't maybe it's not as much of a stretch. Right. Um, but I think that's a real question. Like if you were to ask me, like, if there was by 2030, if there was actual like real demand to build up 3 trillion or 4 trillion of capacity a year, do we even have the power in place to do it? Like, maybe not. Like, probably. Certainly not. The electrical grid probably can't handle that. It was funny, you know, I did a piece of work. Oh God, it was probably a year and a half, two years ago. Even numbers even bigger now. But the idea was I was working with one of my colleagues who covers like the electrical equipment names. And it was like, okay, here's the forecast for Nvidia. And how much would the electrical grid have to grow to support that? And you know, you, you know how this, you build a model and you make a bunch of assumptions like what, Whatever, right? So. But it spits out an answer. I can't remember. I'm gonna make it up. But it was something like, you know, the US electrical capacity has to grow at like 5% a year for the next decade or something like that. So I looked at that, I was like, yeah, okay, that sounds right. And my colleague looked at me like I had two heads. He's like, you've got to be out of your mind. Like, like there's, there's no way. So in his world, like 5% of your is like, is like unachievable. Right. Um, so what does that mean? So there's a lot more things like, like local, like on site power generation has to get done. I mean, they're even, you know, they're talking about turning back three Mile, Three Mile island back on at some point. Right.

Michael Marks: Well, we all know that huge demand creates innovation and uh, there's going to be innovation in the energy side just like there is in all these other things.

Stacey Raskin: It's a good. I never, I never discount human ingenuity. Like, engineers are smart. If there's a business case, a reason to get something that they'll figure out a way probably.

Michael Marks: Right. I think that's right. Any kind of a last, last orientation, you know, we're a venture firm over here. Any holes in the market that you see that need more entrepreneurs to get in and innovate in?

Stacey Raskin: I don't know. I mean there's always. Yeah, I'm not sure.

Michael Marks: Probably, um, I think most of them personally are around these areas that, you know, like, like, you know, reducing energy,

Stacey Raskin: you know, you know, like what I'm, what I am just happy about though. I am happy to see more and more semiconductor startups because for like it was funny, like 10 years ago, I sat down, I started to do a piece of work on like VC in semiconductors and I shelved it because like, there wasn't any. Almost all of the VC work at that point they were like corporate VC or you know, you have like intel capital. That was where it was all being done. There was, there was hardly any because it cost a lot of money to design a chip and a lot of time. And like it was, there was. It was much easier to build some sort of SaaS startup or something.

Michael Marks: Right, Exactly.

Stacey Raskin: I'm really happy regardless of what happens to all these startups, I'm actually really happy to see a lot of like startup innovation happening in Silicon again. It's about time.

Michael Marks: That's kind of the way we feel. We've been investing this for years, so. All right, last question. So if we have you back on the show a year from now, and I really hope we do, and the world plays out roughly the way you expect, what's the industry look like? Who's won? Who's stumbled? What did you get right? What did we get wrong?

Stacey Raskin: Yeah, I mean, hopefully it'll be bigger. Better be bigger. Yeah, I mean, you know, better going into place and I mean I'd love to really. I'm very curious to see like, like where the inference demand and the revenue is and frankly like more and more workloads. What kind of workloads are really getting adopted and ideally like, not, not so, not even so much by like, like engineers, but just like you and me, you know, are normal people actually starting to use this, like, in a bigger way? In ways that are, in ways that are, that are, that are increasing, like, value in their life? I would love to be able to see that in a year. So, like, we'll, we'll see.

Michael Marks: Okay. Well, as I said, we'd love to have you back and it's been great to. So much energy, Stacy. Just great to see, I have to say.

Stacey Raskin: So I've been doing this 18 years and my view has always been as long as I've ever done anything, it's like, I'll do it. It's not fun. And then I'll go find something else to do. I can always find something else to do, um, every single day. Like, I love this industry.

Michael Marks: I do, too. Well, thank you very much for joining us. It's really been great. Thank you.

Stacey Raskin: I'll do that. My pleasure. Anytime.

Michael Marks: Okay, bye. A few things to take away from this one. While everyone has been watching Nvidia, the biggest stock moves came from the bottlenecks. Memory, Optical, the equipment makers. In a supply constrained boom, the leverage sits wherever supply is tightest and that bottleneck keeps moving. The second is that the constraints are shifting from the chips themselves to the physical world around them. The hardest problem in AI may no longer be making the silicon, it may be powering it. US data centers used about 4% of the country's electricity a couple of years ago. By 2030, that could roughly double. And around 2 terawatts of new power is already stuck waiting in interconnection queues. And the last one is the question nobody can answer yet. Whether the revenue catches up to the spending. Everything in this episode runs on the bet that it will. Thanks for tuning in to the Tech Surge podcast from Celeste Capital. If you enjoyed this episode, feel free to share it, subscribe or leave a review on your favorite podcast platform. We'll be back every two weeks with more insights and discussions of all things deep tech.