AI CEO Series: Dr. Varun Sivaram

Speaker A: Welcome to Next in Tech, an S and P global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry Research at S and P Global. And this is the next in our AI CEO series where we're looking at places where AI is being put to work. With me today is Dr. Varun Sivaram, founder and CEO of Emerald AI, formerly the group Chief Strategy and Technology Officer of orsted, a Fortune 500 energy company, a physicist and Rhodes scholar. Welcome to the podcast, Varun.

Speaker B: Eric, thanks so much for having me.

Speaker A: It's great to have you on. We've talked about a number of different applications of AI and where it's being put to work. But this is a particularly interesting intersection that actually hits a number of the things that we've talked about on the podcast, both on data centers, energy consumption, grid operation, and a lot of those interconnected challenges that we face really in helping to manage getting sufficient energy where we need it, when we need it, to be able to manage AI demand, but also to more flexibly work with data center build outs and to be able to meet what is that ever expanding need for capabilities. But what you've been doing is actually leveraging AI capabilities to be able to help manage some of that. Couldn't you give us some details on some of that?

Speaker B: Absolutely, Eric. Look, I think there's a collision course today between the AI revolution and this unprecedented historic growth of AI data centers and what the energy grid is really capable of delivering. And because of this collision course, because we can't get enough energy to the AI revolution, we risk the United States competitiveness and innovation in the AI race. And we also risk raising energy bills for communities and making AI a historically unpopular topic. That's not good for anyone. It's not good for the AI companies, it's not good for America, it's not good for communities. Now, today the United States wants to add, uh, 50 gigawatts or more of AI data centers just in the next three years between now and 2028. And you might recognize that number, Eric. It's because I'm quoting S and P global statistics. But between now and 2028, just half of that capacity in AI data centers can actually get connected to, to the grid, 25 gigawatts or less. So that's a very difficult position that we're on and we need a serious course correction in order to achieve all of the goals. Get enough AI data centers connected as fast as possible, keep bills low for communities and grids, reliable and advance our position in the AI race.

Speaker A: If you think about what the depth of current interconnect queues are for the grid, we're seeing such tremendous delays in, in order to both bring on new capacity as well as to interconnect new data centers to the grid. That presents some, I guess, what seemingly are, uh, insurmountable challenges, it would seem.

Speaker B: Insurmountable, Eric. And the reason for this is you can think of the power grid as a highway. This highway that you probably take your morning commute on has rush hour at 8 or 9am M. Right. In fact, the grid is a highway with horrible utilization at half rush hour, maybe once or twice a month. The rest of the time that highway is relatively empty. There's plenty of room. That's how the US's electricity grids function. There's plenty of spare capacity 99% of the year. And so because the, uh, grid isn't able to serve new data centers for 1% or less of that year, they make a wait, as you said, for 10 or 12 years in that interconnection queue. And that's what's holding up the AI revolution.

Speaker A: Well, that really speaks to the challenge of design of the grid, right? Which is that issue of grid capacity has to be able to meet peak demand. That's the design criteria and that's really what's been guiding design principles. And as you're pointing out is one of the things increasing the time to get connected and the way in which we manage grid capacity broadly.

Speaker B: Exactly. But I would flip the paradigm there, Eric. Uh, and I would say this is not just about grid design and about how you operate the grid. This is about how the users use the grid. Because if there is 99% of the year, plenty of spare capacity, wouldn't it be great if users could just use that spare capacity? Wouldn't it be great that during those one or two rush hours a month, instead of an 18 wheeler, you could shrink that 18 wheeler down a little bit. That is what flexible data centers represent. And that's what Emerald AI, our company, is doing. We have developed technology backed by Nvidia, backed by seven Fortune 500 companies. We've developed technology to help data centers to control their power demand during these rare rush hour periods. And if data centers can flexibly modulate their demand in response to those rare moments of peak grid strain, you could fit a 100 gigawatts worth of data centers on today's power grids right now. So again, recapping, the US wants to connect 50 gigawatts in the next three years, we can only connect 25, but really there's 100 gigawatts of capacity that we could connect right this second if we had flexible data centers. That's the opportunity. It's $4 trillion of AI data centers. And the good news, every time you connect a flexible data center and you avoid having to build out another lane of that highway that's already a massive superhighway, most of the time you save customers money. Local communities will actually see their power bills go down and not up with more flexible data centers. And policymakers are taking notice around the country, states and even the federal government and the Trump administration saying we should really find a fast lane for these flexible data centers. Because if you're flexible and you're going to bring rates down and you're going to make the grid more reliable and you're going to use all that spare capacity where without having to build out extra lanes of that highway, you should get a benefit, you should get a 10 year head start in getting connected to the grid. And that's what we're seeing. Just June 18th we saw a major new Federal Energy Regulatory Commission announcement. That flexible, large load, flexible AI data centers are now potentially going to have a benefit. And Texas has passed a similar rule to give flexible data centers a fast lane onto the grid.

Speaker A: But that's a complicated problem. The data center design philosophy has always been that we're going to be able to go meet demand for our full capacity all the time. And there's been this disconnect between both the power demand side and the workload automation side of what do you run and where do you run it. And it seems like that gap has been pretty vast. There's some specialized applications like crypto mining. You take a look at how crypto miners have set up camp in various places, not only electric grid, but also in natural gas environments where they're looking at stranded capacity to be able to manage that. But crypto mining is one of those workloads that you can start it, you can stop it and shut it down. Uh, typical data center workloads though, there's no coordination between when that workload runs and what power demands look like. And it seems like in most data center environments that there, uh, isn't even a mechanism for it. You look at most cloud providers, you got spot markets to be able to take advantage of spare capacity. But there's no view from the workload perspective into what the current availability of energy happens to be. How do you bridge that Gap.

Speaker B: Eric, that's such a good point. You're absolutely right that making data centers flexible and controllable power users sounds like an alien concept. And you're right. Bitcoin miners can do this. But it's relatively, relatively trivial to ramp up and down how much Bitcoin you're mining. Traditionally, with data centers where you might be running PayPal and Venmo transactions, or a human maybe controlling a surgical robot, you really don't want to mess with the availability and speed of any of those transactions. And so traditional cloud computing has been a terrible, unsuitable candidate for this power modulation. And if you just tried to slap a bunch of batteries onto a data center, it would be an expensive way to, in order to modulate the power drop from the grid. But let me tell you that, uh, although this sounds alien and hard and something that data centers may not want to do, compare it to the alternatives. You realize how attractive this solution is. The alternatives for data centers are you continue down the current path, you wait 10 years, and, oh, in the meantime, the grid has to build out additional lanes to that highway and raise everybody's power bills and, and then you've got pitchforks coming out from communities saying, we don't want you here. Or another alternative is you just say, I'm going to go off the grid. It's technically complicated and expensive to build your own natural gas plant and your own micro grid. You happen to raise prices on everybody else in the process because you're buying up all the supply of electrical equipment and then you've got to, uh, maintain and run your own little electricity grid when right next door there is a massive power grid that is built literally to do that function for you or you go into space. And space is great because there's a lot of literally space. However, it comes with its own engineering challenges of, uh, getting the data centers to space and operating them and using photovoltaics and doing heat dissipation, et cetera. All of these are an order of magnitude more complicated than making, uh, the data center power flexible, making an AI data center controllable in the same way that a Bitcoin miner has been controllable. And I believe that the rise of the AI factory, these specialized data centers filled with GPUs or other accelerators, are actually purpose built to be exquisitely controllable because they do one thing really well. They convert watts into tokens of artificial intelligence. They may be running training or fine tuning or inference. And Emerald AI has built a series of AI tools, AI agents that will, in a very intelligent way, ramp up and down the power by orchestrating these workloads, by keeping the critical ones running at 100% while slowing down and throttling some of the more flexible ones, or by moving certain workloads between locations at the speed of light. So that in Virginia, as we demonstrated with Oracle, you can reduce power when the grid needs you in the middle of a winter peak period by moving them to Chicago during a period where there's excess grid capacity in Chicago and excess data center capacity. So there is technology to do this. Yes, it's complicated, and no, it's not more complicated than the other alternatives.

Speaker A: Uh, so it seems like part of this is really inverting the equation on this. You're talking about moving workloads. This is something where you're overcoming grid inflexibility. Because the challenges we've talked a lot is that the grid has a really hard time distributing energy. It wasn't really built to move energy across long distances. It was built to be relatively universal or relatively unidirectional from generation out to consumers. But it sounds like where you're headed with this is that you're able to move workloads to places where they're needed, and you've also got an understanding of workload characteristics to be able to manage that. Is that where you've headed?

Speaker B: That's exactly right, Eric. And the reframe that I'd say is think of AI not just as a liability on the grid, a big energy suck. Think of AI as the next great energy technology. I was at S&P's Ciro Week earlier this year. I spoke on the main stage, and we made a big announcement with the six biggest American power companies and with Nvidia. And we said, think of AI factories, these new specialized AI data centers, as a living, breathing component of the energy grid that supplements it. Because as you said, Eric, now suddenly you've got an asset that can move the use of electricity. AI workloads around the country and around the world at the speed of light or in one single location. You can throttle the use of energy up and down on command. It's exquisitely controllable. It's virtually controllable. And that's the advantage of these AI factories. And if you think of them as an energy technologies, as virtual generators or virtual power plants and batteries, we can use this network of AI not just as an energy suck, but as a real source of flexibility for the existing inadequate energy grid. So we've done now five demonstrations at, uh, real Commercial data centers all over the world, whether it's London or Phoenix, Arizona. And we've proven with Nvidia and with their partners like Oracle and with the local utilities like the National Grid in the uk, we've proven that uh, AI factories can flex on demand just like a bitcoin miner. You can turn the dial up or down based on what the utility needs and the data center will precisely follow the signal because we understand the workloads and we understand the constraints of the ah, AI factories. Now we're going to the commercial phase. We're less than two years old but we're really excited that within those two years we'll have taken a brand new energy technology from invention all the way through commercialization. A nuclear reactor, which is another kind of energy technology, takes decades to go from invention to commercialization. But this one, AI Factories, is an energy technologies because it's software defined and controlled by AI, we can move at the speed of light. So we're about to do our first two commercial scale deployments. The first one's in Santa Clara with Nvidia and the utility Silicon Valley Power there. We're going to increase the capacity of the Nvidia data center immediately because Silicon Valley Power will reward them for being a flexible power user by giving them additional capacity. And the Second one is 100 megawatts, a large scale multibillion dollar facility. A data center, Digital Realty and Nvidia are building in Manassas, Virginia. And at that Digital Realty data center we're going to be working very closely with Nvidia, uh, to make it a power flexible AI factory that can respond to the local utility and the local grid operator called pjm. The goal is, Eric, going forward, the next thousand AI factories, all built to the Nvidia spec or reference design, will have this feature, power flexibility. So you can control their power use up and down. And a utility anywhere in the world can see the sticker that it's got emerald inside and say, ah, I really want to connect that one 10 years early because I want to have that data center on my grid. That's the one that lowers my customers bills. That's the one that helps keep my grid alive. The last thing I'll quickly say, Eric, is in London. When we did a demo, we proved that at halftime of a soccer game when all the British people turn on their tea kettles and there's a 1 gigawatt spike of energy on the grid, the data center can actually help to stabilize the grid by reducing its power use. That was a really cool and fun demonstration because these AI factories are now going to become the most community and grid friendly assets. They're going to go from villains to heroes. And it's because we've reimagined what a data center is. It's not just an energy suck, it's an energy technology.

Speaker A: Now that the World cup is upon us, there must be a lot of electric kettles kicking on and off. With all the games that are going on, I guess the demand is all that much greater. But I, uh, wanted to touch on one of the points you'd made about battery energy systems and energy storage, because this is one of those things that's been hung out there is all we need to do is have batteries as buffers. But we seem to see a whole set of cautionary tales about not only the limitations of battery energy storage systems, but also some of the durability. We were talking actually a few episodes ago about the challenges that data center environments seem to be seeing just simply because of the wear and tear on the battery environments that seem to start poking some holes in the. I guess that sort of panacea has become the battery idea.

Speaker B: Yeah, absolutely. Well, look, first the last thing you want to do is have a really polluting diesel generator running all the time in order to provide you flexibility. The community will absolutely hate you for that. Then if you start to use batteries, there are a bunch of technical constraints that come into play. First of all, it's expensive, right? If you have a battery set up and you want to make sure that this data center is very flexible so that the utility says to you, okay, I'm willing to connect you immediately and not make you wait 10 years. Then they're going to say, when I run into a difficult condition, we might have to ask you for four or six or eight or 12 hours of flexibility. And a battery just can't do it. And so you're going to have to build a very expensive, massively overbuilt battery to handle all of the different contingencies for which the grid might ask you to be flexible. By far the most graceful, lowest cost way to achieve flexibility is by modulating your workloads, right? The virtual AI jobs that you are able to orchestrate by slowing them down, by pausing them, by moving them somewhere else, that is the most elegant way to solve this problem. And you can actually do it in close collaboration with the battery. You can imagine you've got a two hour battery sitting on your site and you have some flexibility in your AI workloads. And, and you coordinate the two. Emerald has a product that does this. In fact, we collaborate with some of the world's biggest energy companies so that the local energy equipment at the data center can be coordinated with the AI virtual flexibility that our AI agents are managing. That that's the best way to minimize the expense of your equipment, your capital equipment on site, while also precisely meeting what the grid needs so that the grid can give you an earlier and a larger power connection. Today, there's no more valuable currency, there's no coin of the realm, to use a Game of Thrones analogy, than time and speed to power. Emerald delivers that for our data center clients. And we hope that alongside these amazing new commercial deployments with Nvidia, with Digital Realty in California and Virginia, we get the chance to work with customers all over the world. We're now expanding into the United Kingdom, into Japan, into. Because every country needs this. They need data centers to be heroes and not villains, to be energy, controllable assets, energy technologies. And, uh, if we can continue to deploy that, we'd love to see far more data centers built on existing power grids than the other way around. Far more existing grids needed just to power a small number of data centers.

Speaker A: So it's interesting to see some of what you're able to accomplish around that you're really managing demand side. I'm curious about how you're putting AI to work. And is that scheduling, is that management? I'm curious about what aspects of that you're taking advantage of and what you've seen to actually get real results.

Speaker B: It's a great question, Eric, and it can be a little confusing because we're almost a recursive AI company, right? We are AI managing AI. And so we think about AI foundationally. We are a frontier research laboratory for AI and energy. You can think of anthropic as a frontier research laboratory for AI. We are the frontier research lab for AI and energy because we're doing foundational AI research. All AI workloads, or almost all of them, have some amount of flexibility inherent in them. If you're fine tuning a particular model, that workload might have some temporal flexibility. You might be able to slow it down a little bit. Uh, if you're running AI inference, it can be batch inference, which has some temporal flexibility or serving inference. You need the answer right this moment. That might have some spatial flexibility. You might be able to move it at the speed of light to another data center. And there are hundreds of different AI workload types. And we work on Understanding what are the different flexibility parameters for each of these AI workloads. We've used AI to train our own model in order to quickly and rapidly pattern recognize what sort of workload is running on an existing data center. And we have to use AI agents in order to orchestrate dozens or hundreds or thousands of AI workloads running at one data center or across a network of data centers. No human could make the decisions that we had to make when in London, a lightning strike hit and we had to have the AI factory respond within seconds to reduce the power consumption by over 30% in under 30 seconds. And so the AI that we use is orchestrating the AI that's running on the data center. And we've published our research now peer reviewed platforms around the world, like one of the world's top scientific journals, Nature Energy. So we have real results and we are at our heart, a research laboratory, an AI frontier, research lab for AI and energy.

Speaker A: Oh, wow. So you are taking what you understand about the workloads, about the dynamic nature of the grid, what the capacity capabilities are, and being able to in real time, do that scheduling so that you really have that impact on grid demand at any point in time.

Speaker B: That's exactly right. And so utilities look to us as their best friend. Utilities need to look at AI factories with Emerald inside as a real resource. If you're running into a pinch, the AI factories can help bail you out. Now, we don't want to do this too often because AI factories, I want to be clear, run the world's most valuable workloads. You might think that generating cat images from ChatGPT is not very valuable, but this is. Whether it's that or you're creating slides or you're doing coding work, knowledge work is going to become the most important part of the global economy. So we don't want to take these assets for granted. We want to judiciously flex them when it will not disrupt the core mission of those AI workloads. But, uh, because they are exquisitely controllable and our AI agents are capable of controlling them in such a way as to protect the customer's experience while also precisely meeting grid targets, we believe that this can become an indispensable resource to the electric power grid.

Speaker A: Well, uh, as you're pointing out, those that are looking to get their own version of astronauts riding horses on the moon might actually be able to accept a bit more of a delay, especially when that gives us better demand profiles and capabilities to actually work cooperatively with a grid. Um, those have been fascinating. Thank you.

Speaker B: Varun Eric thank you so much for having me. Really an honor to join S and

Speaker A: P. Thank you very much and especially in such an important topic. But we are at time for this episode. Thanks to our audience for staying with us and thanks to our production team including Sophie Kaur, Ferranmi Daeshan, Kira Smith, Dylan Scheibel on the marketing and events teams. If you like this episode, please subscribe or like us. Please keep in mind that statements made by persons who are not S and P Global employees represent their own views and are not necessarily the views of S and P Global. I hope you'll join us for our next episode because there is always something next in tech.