Bridging Physics and AI for Smarter Climate Decisions | EP. 44

Welcome to Hidden Layers, where we explore the people and technology behind artificial intelligence. I'm your host Ron Green. Today on Hidden Layers, we're joined by Dr. Hannah Liu, assistant professor at the University of Texas at Austin, and core faculty at the Odin Institute for Computational Engineering and Sciences.

Dr. Liu is helping pioneer the use of AI to make complex simulations faster and more accurate. Her research plans, scientific computing, reduced order modeling, and uncertainty quantification, with machine learning to push the boundaries of what's possible in environmental fluid mechanics and earth system modeling. Her work is building a new generation of tools for understanding our planet and making smarter, more informed decisions about the climate and the environment.

Alright Hannah, thank you so much for joining us today. Thanks for having me. So you are at the University of Texas and you're pioneering the use of artificial intelligence, machine learning models, to basically speed up simulations. And you've used this really successfully, and I'll link to the paper on the podcast, but you've used this to model sort of CO2 absorption geologically.

And all the complexities around like fault lines and all the very non-linear complexity of geological deformations, etc. Historically these types of simulations have been done using physics-based software where you have equations that theoretically capture the interactions of all the gases and solids, etc. But they're so expensive computationally, right? And so you've been exploring using artificial intelligence to speed that up computationally.

Can you talk a little bit about why this problem is so difficult, why traditional methods are just so computationally expensive? And then I want to segue into your work on serial models. Okay, that's great. Yeah, so you know, like surrogate models, also sometimes we say reduce our models.

They are simplified models for complex system, usually very expensive to simulate. You know, for physics problem in science and engineering, we rely on partial differential equations and simulating them a very time consuming and super expensive. And thinking about, even nowadays we have more advanced computational capacity, it is still very challenge to simulate thousands, millions of them when you consider answering quantification, optimization, control problem. And in many applications, we have to make real time decisions based on the model predictions, which cannot be supported by expensive traditional simulators.

So there have been a variety of surrogate models, some of them looking for simplified physics, some of them looking for a projection from the high dimensional space to the low dimensional space. And with machine learning, deep neural network, we also have this deep learning base surrogate models, which is trying to figure out the relationship between input output in a physical system by leveraging the structure of the data from simulation or from experiments. Okay, all right, so that's a perfect tip.

So surrogate modeling, which I'm relatively inexperienced with, so I'm pretty excited about this. Surrogate modeling, what is it at a high level? How is a surrogate model different from a traditional simulation? How is it trained?

Why does it work? It seems almost like magic. Yeah, so that's a good question. So first of all, it has to be much faster than traditional simulators.

That's why you want to use it. That's the main thing it's bringing to the table of speed, maybe. Yeah, especially as I mentioned before, the applications where you need thousands, wrongs, meetings, wrongs for optimization for real-time applications. Right.

Yes. Okay, and so you have these surrogate models, it seems almost magical to me. You're saying, hey, we have this really expensive computer simulation using equations that we think accurately model some complicated process, but it's too expensive to run. So we're going to go with this other approach that's simpler, but you can still get accurate predictions.

When these models are trained, how are they trained? Meaning, are they taking in exactly the same information that the traditional simulations would work with, performing a variance of the computations? Or are they looking at less data, or just key parts of the information that would go into your traditional simulation? Yeah, so in the big picture, as you said, we actually have two phases for surrogate model.

One phase is training, the other phase is using it for predictions. In the training phases, we first have to collect data from experiments or from high fidelity simulations. So the data set usually consists of pairs of input output. For example, the input can be a scenario where we want to reduce carbon emission by 30%.

And the output can be what is the global temperature response in 10 years. And this input output can be generated by very expensive global climate models. It can take days to simulate this kind of stuff. And after you train your network, which basically is solving an optimization problem of identifying all of these hyper parameters in your network, which minimize the difference of your model output and the reference output.

And after you train it, you can have a totally new input. Let's say we cannot afford 30% carbon reductions, but we can do 20% reductions. The model can predict what is the global temperature response corresponding to that. And then we can make decisions based on these kind of scenarios.

So it allows us to explore different what if scenarios and help us to quantify the uncertainties associated with it. Okay, that helps a bunch. So when you're training the model, are you training it almost exclusively on the input output from the more computationally expensive simulations? Yeah, depends on what is your interest of the problem.

It can be your, it's basically learning relationship between the input and outputs. It can be a state to state regression problem. It can be a classification problem. It can also be a time series.

Right. So depends on how you define your input and output. Okay, great. We've talked about Dr.

Steve Brunton at the University of Washington. We both really love his work. And he's using, or he's a big advocate of what he calls, you know, physics-based machine learning or AI, where you essentially, instead of taking a model and letting it really sort of be unconstrained, let's say in the output space, you know certain relationships, or you understand the physics of the environment involved. And you can actually apply constraints to the model using, you know, known physical laws.

My interesting is, in your work, you didn't do anything like that, right? You guys left the model mostly unconstrained as that right on its prediction. So does, does Steve work, like, does it impact your thinking, like, how, how are you thinking about the future of surrogate models? Are these two different paths, or do you think they're related a little bit?

Yeah, so I'm absolutely a fan of his work, his research, his YouTube channel, and the magic. He's so good. Exactly. Looks like he's writing backwards.

It's amazing. Yeah, I mean, actually, my PhD thesis is on reducing the modeling start with reading his book on dynamic mode of decomposition. And just this summer, I also passed the same book to one of my students, you know, Steve Brownton, he's definitely an inspiration. So there are a lot of work about how to use machine learning to physics problems appropriately, and of course there are many ways.

But we all know that machine learning arrives from computer science, where, you know, the application is more for, like, image classification, where you have large amounts of data, but you don't have any physics or equations here. Exactly. So it's true that nowadays we have more data than ever, but when it comes to complex physics problems, real life, engineering problems, first of the data is always indirect. It's like I can get my heart rate measurement from my Apple Watch, but it is just an indirect measurement of the complex dynamics of my heart.

In earth science, we can also get pressure readings from a handful of wells, but the true dynamics of the subsurface is more high dimensional and uncertain. And also the data is always sparse in space and time, you know, you do not want to live in a world where earth is literally literally with sensors. So the data is always sparse, noisy, indirect, and that has put a lot of challenge for classical machine learning. But luckily we have centuries of studies in physics, you know, like Newton's laws conservation laws, and in different ways, we can use the help of these physics laws to help us construct target models, which can work fairly well, but with relatively small data set.

I think that is a goal, and there are different ways to do it, like using the PD as a penalty in the last function, or building a library of possible terms in your equations that is more of the single method developed by Steven Brownton. And one of my previous work, maybe related to dynamic mode, the conversation, which is a pure data driven method. It doesn't work very well for a physical system with a deduction dominated system. But when you combine it in the construction with the advection speed, which is a meaningful physical quantity in the system, when you incorporate this in the formulation of your target model is super helpful in terms of improving accuracy and the interpretability.

Okay, what occurred to me right then was if you're using a serial model on a system that is maybe like chaotic, where really small changes to the input could radically change the output. That seems to me like they would be a bad candidate, right? Are you seeing anybody using these target models with chaotic systems, or is it more of a trend to avoid those for that very reason? Yeah, that's a very good question.

I mean, even without thinking about a chaotic system, if you just think about the super nonlinear system, there has been a lot of challenges here. I think literally, there are many work in terms of quantify the error between a target model, certain type of target model, and zero corresponding physics based models. In this setting, everything can be written down precisely in equations, and I also had some previous work on this. We were able to do some mathematical analysis and derive some error bound for these systems.

We can even say something about when they are reliable, when they will not become reliable. But for general system and for target models, especially constructed by deep neural networks, it's very challenging to derive such kind of error bound and do this analysis, mostly because our theoretical understanding of deep learning is still on the development. But I think a more challenging and important thing is all of this framework assumes that the physics based models is a ground choose, right?

Which is often not the case. I would absolutely imagine in most cases it's not. Because these physics based models, they are themselves approximations. They rely on simplifications, approximations, and also they are inherent with uncertainties from material properties, geometry, boundary initial conditions.

So when you think about the issue of trust for models, for target models, it's not only for target models, it's actually the fact that we should be aware of all of the uncertainties comes with all models. So I think we should be very cautious in terms of using target models. And do you comparison with high fidelity simulations whenever you can and validate with experiments, experimental data whenever you can. But still it's very helpful and meaningful if we can use the target model to guide us, investigate where will be the most informative region to do your next expensive simulation or expensive experiments so that we can test the boundary of our models.

Let me say that back to you. We've got these surrogate models, which are taking in inputs that are actually the results of a more of a classical physics based simulation, those inputs, which are the outputs essentially of those classical physical based model models. They may not be accurate themselves right there, they're roughly accurate so you know it's it's almost like you're compressing something that's already been compressed in a way right. But you can then go back and validated against what happened in the world world as those simulations get better.

Okay, so this really, this is one of the big questions I was dying to ask you today because I'm really blown away by these world models that are being created in AI right now. For example, the new VO3 model from Google where you can you can prompt these visual video generation models and they will produce output that is shocking or realistic right I've seen. I've seen examples of you know a prompt was like to you know to old wooden ships in a coffee cup fighting a battle right fighting a sea battle and what it's amazing about that is you see.

The waves like the fluid dynamics look so accurate and if you look at the you look at the light it's almost like there's a ray tracing algorithm running except for we know that none of that was put in the model seem to have learned some. Much much more compressed way of computing outputs that at least seem realistic to us right we I don't know if they if they are actually realistic but visually they seem to work my thought there is it is is that essentially almost the same thing is the work that you're doing certain models where you're you're learning maybe a compressed manifold on which you can.

You can't simulate everything but you can make predictions about the sort of the most important factors that you care about in that system. Yeah so you're talking about diffusion models and maybe more broadly speaking generative models. There are definitely one of the most popular research topics nowadays I think I think they are you know the idea of compress the complex system high dimensional system into something more smaller and faster to evaluate is quite a line with certain model.

And I think what makes division model particularly exciting is that they are not only model try to make one predictions for one point but they are try to model the distributions right so in principle they can. You know not only make one fast predictions but also for example generating ensembles of different plausible scenarios and help us in these what if questions and also in quantifications. But again the question is you know different from video generating we cannot just generate a flow field that looks right but validate like violate physics laws like conservation of mass.

So the question again is how do you enforce or how do you make this power for generative models also respect all of these physical constraints and I think that's a challenge and also the research opportunities out there. So a few more questions for you I'm just so fascinating so where do you see your work going in surrogum models like do you think that this is. Do you think that this is really going to be it feels to me like core to simulations going forward we're just having so much success on so many fronts learning these these more sort of compressed.

Representations these high dimensional spaces and enable to do so much more so much more quickly are you really excited about the future here yes I am I am personally especially interested in. Address all of the uncertainties comes ways the target model or you know existing physics models and bridges a gap between whatever models we have and the reality which we usually obtain by observations or measurement data so it's like. You know I don't want to commit myself in finding the perfect model right because we have to embrace this idea that the the model might fail sooner or later or in different conditions so instead we should kind of develop a model that can adapt can do improvement.

With incoming more information about the system with incoming more measurement data from the system and so that we can make better predictions for the next step and make better decisions from that I think this is where you know this called model calibration comes in and also very aligned with. This new concept of digital twins you know it's really these by directional interaction between. A virtual model and a physical access is a key there to make higher fidelity of your model right right do you.

There's a lot of interest right now in biology around digital twins you know there are lots of companies out there trying to create you know sort of the simulations of individual cells and things like that i'm wondering about about going the exact opposite direction like. Is there much work being done in taking. For example surrogate models and trying to is drive formulas that are sort of lower dimensional representations of the system that are more computationally tractable by that I mean obviously these are models are performing some type of computation right.

In the work in your paper you know they're relatively small you know fully connected neural network so it's essentially just mostly linear algebra with some non linear functions but you i'm wondering if there's a way to. Like take that and then drive a more efficient sort of analytic solution that you can then go drive the simulation. What do you i'm is that making sense you're going with that can you see more about it yeah yeah what i'm what i'm imagining is we've got these we've got these physics based simulations that are computationally really expensive.

And you've got the surrogate models which are much much more efficient i wonder if there's a way to once you have a surrogate model that is accurate enough use it to essentially drive. A better more computationally efficient set of formulas that you could use for the simulation and you wouldn't even need the surrogate model at that point I see I see a point so that is more I think related to kind of. Quantity of interest driven learning that is kind of that is a direction of you know sometimes discover maybe new physics here yeah that is also a fantastic area you know.

Use the surrogate model use the data modeling tool as a way to discover the new physics yeah discover the formulas that would my that would. Allow you to represent it in a lower dimensional space essentially yeah that makes sense okay that's great can you show us an example of the simulations what they look like I know I know you've got some some video of the computer simulations and then the outputs from the surrogate model itself. Yeah so in the left corner you're saying a video recording of an experiments that's done in Norway my collaborators is just building this beautiful tank with layers of different geologies like different layers of sand to mimic the geology formations and they inject CO2 at designed locations.

And this as you can see it's super non-linear system and with fairly complexity of the first geometry and if you see there's some beautiful dissolution fingers that is due to hydrodynamic instability. And of course we we have physics based models we have the equations to solve in the middle it is showing here. But this equations comes with a lot of uncertain parameters so here in the bottom the blue lines is showing some ranges of the uncertain parameters. Here is just a three but we have a lot of them in the system so each combinations of them will give you very different solution profiles can be very different from the experiments.

And of course one of them there exists one of them within this range that can produce the simulation solution profile match with experiments very well. And here is my collaborator a real human who identified this one combination in the primary space that can produce this beautiful simulation that matches with the experimental results but it takes so long for it. Each round is like hours to days to finish and you have to write multiple times so that you can match with the experimental data so it's super tired for super exhausting procedure.

But today if we you know use the simulation data and train a machine learning model a surrogate you can just like within milliseconds you can produce a fairly equivalent predictions and match with experimental results very well. But more importantly we can use this chip surrogate to help with us in this parameter identification procedure and here you see these gray bars of histogram. It is not giving you one solution of the possible parameter combinations but it gives you a distribution of the possible combinations.

And some of them as you see in the first figure it is agreeing the machine is agreeing very well with human identified value this red line here. But in some other cases the machine learning is indicating some other possible solutions of the parameter combinations. So it will help us to explore you know the other spaces you know maybe higher values in the intervals here that can give you equivalently good match with the experimental data or even better we don't know that. So that just to propose a lot of possibilities to explore the answer in the system and help us really understand this physical system.

Okay that makes perfect sense. We'll link to we'll link if you're watching this on video you can see this if you're if you're listening to this we'll link in the podcast to this image that people can take a look or talking about. I know this was so awesome thank you so much for joining us today I really appreciate it this is thanks for having me here. Thank you for listening to Hidden Layers this series is hosted by Kung Fu AI a management consulting and engineering firm focused exclusively on artificial intelligence.

If you have any questions or thoughts about today's episode or if you know someone we should feature please visit us at Kung Fu dot AI.