This Just In Radio: AI in Action with Tony Nunes of AMD

Speaker A: Welcome to this Justin the Show, bringing you the latest advancements in healthcare strategy, innovation and public policy. And now, for the fastest voice in healthcare, here's your host, Justin Barnes.

Speaker B: Thank you for tuning in. Welcome to this just in. I'm, um, your host, Justin Barnes. And these episodes, I bring you the latest advancements in healthcare strategy, innovation and leadership. As always, we're broadcasting from this Justin Studios on the Business Radio X Network, the Health Care Now Radio Network, the Global Health Podcast Network, and our newest broadcast partner, the Health Podcast Library. For this episode, my 399th, I have something a little different and special for you. I'm sharing a conversation I recently had with Tony Nunes, Senior Manager of Healthcare and Life Sciences at amd, recorded during the AI in Action Virtual Summit. Our session was titled From Data to Decisions Making AI analytics actually Actionable. And honestly, that title says it all. Tony is a longtime friend of the show, and if you've heard him before, you know he doesn't deal in theory. He deals in what's real, what's working, and what healthcare leaders actually need to do right now to move from AI pilot to practice. This one is packed. Let's get into it. Tony. We'll go ahead and dive in, my friend, and begin. So our session, data to Decisions Making AI analytics actually Actionable. Tony's actually a longtime friend of mine, a longtime friend of my show, uh, and is just a true expert in AI, all things, um, GPUs, CPUs, APUs, but also really seeing AI in action, not just here in America, but also globally. So bringing all those best practices here, we always have great conversations. So let's go ahead and dive in, kind of setting the stage a little bit, the gap between hype and reality. So, Tony, we've all watched AI go from pilot projects, uh, to the boardroom, and seemingly overnight from where you sit at amd, working with health systems like science companies, um, and even AI developers every day. What's the biggest gap between what healthcare organizations think AI will do for them and what's actually happening on the ground today?

Speaker A: Well, that's a great question. There's actually a lot, a lot to unpack in that, believe it or not. So if we just look at the numbers. So in the last three years, there's been almost a feverish adoption of buying GPUs with the idea that this is somehow going to transform operationally, uh, and bringing automation to drive down cost. And that's still the goal that most organizations have. But between wanting that to happen and having that happen are two different things, unfortunately. So in talking to different leaders, and it came up this morning in the meeting I was at, there's a 77% adoption of AI, at least as a science experiment in most healthcare companies. Now when I say healthcare, I'm actually grouping life science and hospitals in this umbrella. But that 77% number is dwarfed by the less than 20% number that actually goes into production. I think the gist of your question is why is right. Typically it's this. So when healthcare has adopted a technology, they think of it in the same way they think of buying a new storage platform or a new compute platform where they simply retrofitted in to the existing workflow and expected to make that workflow better. This is not the case here because it fundamentally transforms not just the speed of the technology, not just the ability to process faster, but, but fundamentally how the people in process integrate into this new model. And I don't think that was fully understood. It's getting better now, but it wasn't fully understood going forward. And I can think of an example. I was in Raleigh, North Carolina about a month or so ago in very large, uh, institution there, you can probably guess who that would be. Ran into this issue and they decided to come up with a very, actually really creative way of handling that and going into production faster. They created a lab environment, a simulation of all the different workflows. And this could, this could encompass uh, uh, medical imaging, this could encompass pharmacy, et cetera. And they, they gained out all the changes that would accrue by having automation from AI embedded into that workflow and then made the adjustments before it went in production. So that when they actually adopted this, they already knew where the benefits were. They already knew where the changes had to be made on the process side and on the people side. And then frankly that's the advice I give everybody. And it's resonating, it's just not easy to do. But if you're going to make the maximum return on an investment in any platform, this has to be done. It has to be, it has to be understood. Yep. It's a bit long winded.

Speaker B: No, it's excellent. I completely agree. All right, well, I do want to keep moving here. Um, no. Great. And we're going to dive into the ROI here in a minute as well. Um, but this is fantastic. Ah, Tony. So when talk about the hardware foundation, there's a lot of conversation around models, algorithms and data, but not enough about it's actually running underneath it all. I know this is near and Dear in your world, Tony, so how much does compute architecture matter when a health system is trying to move from an AI proof of concept to something that's actually deployed at scale and producing real decisions?

Speaker A: There's talk these days, I've run across this a uh, bunch of times in the last two weeks actually where the idea is you don't need CPUs anymore, you just simply get GPUs and it'll do everything that you need. Nonsensical in my opinion. Number one, the cost. GPUs are quite expensive and what they're good at is the training side of the equation being the, being the bucket that allows all the material, all the information with regards to a uh, model that contains medical imaging for example, or billing or something else, to be able to train those models so that you can then develop the use cases against that. What it doesn't do exceptionally well is the inferencing side of this. And frankly at namd we're pretty much you know, focused on the fact that the inferencing side is what's where the growth of this is going to be. Because that really is the brains of what AI, uh, as an entity work, uh, makes uh, is how it works, is the inferencing that comes out of that to be able to create the level of automation that's required frankly to lower long term costs. Because that's what this is all about. Ultimately AI is a mechanism for lowering cost. So it's that balance between CPU and GPU in these hybrid environments and tweaking those for the right level of hybrid performance that's going to make the differentiation in success and failure. And you know, low latency, high throughput, cost efficiency at scale. All of this has to be taken into account between both. I know plenty of uh, startup ISPs that use CPU alone because in some cases it's far more efficient than using a GPU at a certain, at a certain threshold. But as these models grow it becomes a necessity to incorporate both. And I think anyone who's, who's done any level of, of work in integrating these, these AI models will tell you the same. So infrastructure choices will determine whether you're going to get real time insights or delayed reports. Okay. It's the difference between, you know, using Claude effectively or treating Claude like a three year old toddler basically.

Speaker B: Right, great point. So as a follow up there, you know, we do talk about CPUs, GPUs, even APU's sometimes, but we're in CPUs and GPUs the right tool for the right job. Help our audience understand this practically, what does a healthcare AI workload actually need? A GPU versus a high performance cpu. And are organizations getting this wrong in ways that are costing them time or money or both?

Speaker A: Yeah, uh, basically if the foundation computer isn't designed to scale, AI stays in the science budget. So and that tends to be the same. And ah, the threshold is variable. To be honest. A billion, a billion factors or a billion variables is easily handled by a CPU. And the rule of thumb is 7 billion and above. You sort of have to incorporate a GPU into the mix. That's a loose equation. It really depends on what you're trying to do. So I do know some protein analysis isps that are looking for patterns, different proteins that are looking for um, modeling of um, antibodies use a CPU only. Right. And, and that's, and they're, and they're doing putting out fantastic work in that regard as the model widens. And typically what, what happens in these things is a use case is narrowly defined and then expands in scope over time. Um, so if you're, if you're looking, if you're looking at, at a different billing models in Austria for example within the parameters of that institution you could probably just use a cpu. But as you start expanding into the partnerships with the payer side it becomes more complex, the integration becomes more complex and then the different learning buckets um, that accrue from that require the, the extension to the GPU to large GPUs to be able to do that. And at that point this becomes and art more than science in being able to determine what the balance is. And that requires one staff to be able to do that and to the commitment to making it work again if you don't do that then it just becomes, it just stays in the science project. It never, it never, it won't be designed to fail.

Speaker B: So um, quick follow up on that just for our audience is mainly health systems, hospitals and health systems. And it goes back to the use case. But is there a direction that someone would start? Obviously again define your use case and the data and how much and what processing power you may need. But is there a best practice or a benchmark or a bench line that you would give as a recommendation CPU

Speaker A: versus you that is on the use case. It's very difficult to, to, to measure but there, there's white papers that we put out and you know, for anyone to ask, I'll be happy to send these out but it's really dependent on the use case and frankly it's this. So if, uh, the EMR as the nexus for all patient information, which in fact it's grown into, a lot of the different workload models are now embedded into that emr. And so it becomes not a question of that individual workload, but the totality of the different workloads that are being managed by the different agents that determine what the, what the infrastructure is going to look like. And that's different. And that's different. If you're looking at a digital doorway project or if you're looking at a um, clinical project that's geared towards physic versus uh, support, it's all going to be different. But the key here is the adoption of this, at least in the hospital, is really going to be determined by what's available within the electronic health recording system or the emr.

Speaker B: Makes sense. Excellent. So, next question. You and I talk about this a lot over the last couple of years. Data quality, you know, we both know that the most sophisticated hardware in the world can't fix bad data. Uh, in your experience working with healthcare and life science organizations, how often is data readiness, not the AI model itself, the thing that's actually blocking actionable analytics?

Speaker A: Great question. It's like 70% of the challenge. So, and again, I'll go back to medical imaging as an example. My background going back almost 30 years was around medical imaging. And I had always worked under the assumption that if something's at a DICOM bag, that those are accurate and as it's archived that we can use that to be able to build out um, different AI models for image analysis. It turns out not to be the case. In fact, the variance is so wide that less than 10% of the available medical imaging is AI ready to be used. And so we've had to put an intermediary step in between and actually look at the image itself. Forget the tagging that's on there, Forget the.com tagging, look at the image itself, create a new structure outside of that in order to be able to then use um, that archive of images to be used with AI. And uh, that's been one of the biggest surprises over the last year is the lack of quality of data that exists so long before you can determine whether AI is going to be successful or not. You got to look at the condition of the data. Is it in uh, uh, a metadata rich state that can be used and if not what are the steps necessary to get it to that? And I know it increases the overall scope of a project. But if you don't do that, it will not be successful. Again you're going to look at a science experiment for a limited scope rather than production.

Speaker B: Excellent, excellent.

Speaker A: To go back to the previous question, this is where the training models, which GPU become really critical because this is geared exactly for them. Yep.

Speaker B: Nope, great point. And I can kind of carrying on here and this is a little bit on the imaging genomics and the compute intensive frontier. You know you guys play AMD plays heavy in that area. Certainly medical imaging and genomics where walk us through what's happening at that intersection. I know we have that think tank actually life sciences think tank next week. So this may be more for some of the research in the academic medical centers. But what are you seeing at the most meaningful, um, clinical outcomes exchange between high performance computing right now and where these clinical outcomes are going.

Speaker A: Yeah, so great question. So you actually asked a bunch of different questions in that one question. Yes. Um, so sequential analysis, which is what genomics did. Okay. So sequential reads of, of a full genome that's fairly easy to calculate because it's, it's, it's, it's, it's, it's um, it's, it's single format coming through and so a cpu, a single CPU handled that quite well. Um, if you're trying to then compare something like that with proteomics where you're looking at the different reference models that come out of folding proteins or looking at adders. An interesting case I saw recently, uh, creating an agentic model that lets you model in silica what if scenarios, if I take the different factors from protein folding from um, um, patterns that are noticed and if we were to manipulate this, what kind of molecule can be identified for drug discovery? And frankly this one area, this drug discovery area is where vast amounts of investment are going into right now and in fact requires the largest amount of compute. So in our event coming up that's going to be the focus. Exactly. How do we use the appropriate amount of compute power? By this I don't mean just cpu, but a hybrid environment to be able to accelerate both the learning model and the inferencing model to determine uh, in silica modeling, digital twin if you will, on what the next molecule can be for a particular set of factors.

Speaker B: Yep, I love it. Exciting. No, this is a. Ah, hello. We're doing. And um, obviously you guys are out in front of this so you've also now you keep bringing up some great um, positioning for me. So I love the hybrid, the cloud discussion here on prem Um, I know where you are today. It's kind of cool to see some of the stuff come to maturation. So there's a lot of pressure on health systems to go cloud, quote unquote. Um, but for a lot of reasons, latency, data sovereignty, cost. It's not always the right answer for AI inference workloads. What deployment architecture are you seeing actually work in production environments today?

Speaker A: Yeah, this is an old model guys, I mean it seems new. But this was the same argument when cloud, uh, was first available to it some years ago is cloud was going to solve everything, let's just move everything to the cloud. And then reasons of cost and reasons of control mean that not optimal. And so some compute came back and the question became, well, what is it that should remain under our control in our own data centers versus what can we consume in the cloud? That's exactly the same argument that's being had right now. Right. So where do you consume AI? Uh, again it depends. So the strength of the cloud is elasticity in training large models because the beta is there. Okay. But the strength on site in the data center is the low latency. And for clinical decisions this is critical. And then there's a whole question of data sovereignty, especially on the life sciences side in drug discovery, this is, this is not something to be ignored. It becomes a critical component and compliance. And that's something that's better done on prem rather than in the cloud in my opinion. So the question becomes what is the hybrid architecture? And in simple terms, if you need to train large models, train them in the cloud. If you go to infer, do the inference, do them on prem, or in some cases even do them at the edge.

Speaker B: Mhm, that's excellent. So as a follow up, you and I were uh, probably two or two or three weeks ago, we were discussing this and you're talking about the cost of doing some of this on premise now is going to grow significantly and already has. Can you touch on that a little bit? Just because we need to think about this not only where we are today, but think six months from now, one year from now, 18 months from now. Talk about cost.

Speaker A: Yeah, so the current nomenclature around this is two things. Either the cost per wattage, which is the cost, which is an amalgamation of a couple of different factors, but basically how much does it cost me to run, actually run these models? Uh, whether onsite or in cloud, the extension to that is now something called token economics. So as the number of tokens is used because you made these different models more available in Your organization, the cost to use these models increases exponentially. And so it's the same question early on in the cloud, uh, this is just simply too expensive. We used to be a couple of thousand dollars turning into tens of thousands, hundreds of thousands. And he write about an organization, at the end of the day that it's costing them a hundred million dollars to use the model. Clearly this is not something in fact economic long term viability. So the question now what we absolutely have to use in a hyperscaler and scratch hyperscaler because there are now neoclouds that are specializing in this and they can include, I include all of them under the cloud umbrella. Even though, uh, you know, you can argue that there's differences and there are, but let's just say off site versus what you're doing, what you're doing here and you have to go into the taxonomy type, um, what's the most important part of this? Is it the, is it reduced clinician time? Is it automation of workflow? Is it faster diagnosis? Is it reducing error? Is it performance per watt or model? And that's going to be different depending on what you're trying to do. So right now in the life sciences side, for example, on the drug discovery side, the consumption has been in the cloud. I suspect that more and more of that will be pulled out in a hospital setting, um, where a lot of the data is already on site because they control it through their emr, um, um, or it's hosted by any platform. It's probably going to be, it's going to, it's probably going to be 50, 50 in the cloud versus on site. And again that marker can move, uh, back and forth. Um, but you've got to focus, the focus has to be on which one of those things is important to you. Reduced clinician time, automation factor, diagnosis, reduced errors, performance per watt, cost per inference. That has to be undertaken in order to make this a success. Performance per dollar Performance per one.

Speaker B: Yeah, I think this is a really important conversation to have because I've not heard it out in the industry and you really got to think through this as you're standing up these AI models to really understand what you're trying to achieve. So brings me up to my next point. Um, second to last question here, the ROI conversation. When you're sitting across the table from a CIO or a CFO at a health system, what does the ROI conversation around AI infrastructure actually look like? Like what metrics are healthcare organizations using to justify the compute investment?

Speaker A: It's not always easy to measure. And I bring this up all the time. There are no easy drive metrics that you can point to to create your AI at this point. It is how, how has this automation either improved patient experience or improve researcher experience, or improve clinician experience versus not having it? And that has a tangible value, but it's not always measurable. Are we going to have to reduce staff by implementing some of these automated procedures? Possibly. And that's everyone's fear. Everyone's afraid of this, but I think the fear can be exaggerated. Yes. There's going to be changes. Is it going to be wholesale? Okay. In my experience, a lot of these models need to be as narrowly focused as possible, um, in order to be effective. And so the wider use of a larger language model is probably not going to be ideal in most settings. Therefore, the real question becomes will automation make my physicians, uh, feel better about what they're doing? Will their experience working for the institution improve immersively? Will patients have a better experience because they're now using some of the digital interactions in the digital doorway, the digital human interacting with them, um, being their first line of interaction between the patient and the hospital. Will that improve their experience? Possibly, and in fact in some cases dramatically. So that intangible is what has to be looked at right now. Because, uh, until the workflows are adjusted completely to take this into account, it's going to be really hard to measure the roi. But everybody is looking at this, everybody is being forced to evaluate this. So it's not like you can ignore it. You're not good. There is no going back, people.

Speaker B: Okay?

Speaker A: The train is moving and no matter what you may think personally, this is going to happen. So the question becomes, how do you make this effective for you? Is there an roi, a concrete ROI you can point to? In some cases, yes. In most cases, not yet. But there are tangential benefits on, um, the softer side that have to be taken into account.

Speaker B: Absolutely. Here, one last question. Um, looking for a one minute answer because I want you to do a couple of questions, advice for the room. Let's close with this. If you're a digital health company or a health system leader watching the session today and you want AI analytics that are truly actionable, not just a dashboard that nobody looks at, what's the one thing you'd tell them to get right first?

Speaker A: Tony, if your, your definition of work be well founded. Okay, Excellent. Do not try to boil the ocean. Use a smaller set of data and a smaller tangible use case that you can apply. Work with that, refine that take those learnings, go on to something else.

Speaker B: Excellent.

Speaker A: If you can do this consistently, you have built out 20, 30 different use cases that are effectively using automation. Three I'll be able to do and then evaluate. I mean look, you don't point technology in a vacuum. It touches people, it touches process. Absolutely. Evaluate those, those touch points and then go forward. Those are the most successful, uh, deployments of this that I've seen and there are many actually. Use some, um, example of what I just described.

Speaker B: Excellent, Tony. Appreciate it, my friend. One question just came in. We've got about a minute left to the session. Are LLMs heading to the equivalent of on prem behind the health system's walls as a means to prevent phi leakage, uh, to the public space?

Speaker A: LLM. But language models, typically they're going to be smaller and defined within, within the sovereign bounds of your institution. Yes. Okay, absolutely.

Speaker B: Excellent. Probably have time for one more question, Carol, if you have any. If not, we'll let Tony go catch his plane.

Speaker C: Just an overall discussion is an overriding question.

Speaker B: If we have a second. Do you want to read the question?

Speaker C: It says, it says an all. This discussion has the overriding question of who needs to give their consent for the use of their data. Are health systems actually, and actually all healthcare providers obtaining informed consent from their patients because people have been able to be identified from de identified data using multiple DE identification data sets and the use of AI?

Speaker A: I think I understand the question.

Speaker B: Yeah.

Speaker A: So the question becomes, ah, who owns the data? Yeah. Ah, do I have to get consent to be able to use even a DE identified model? Um, yes, in some cases, yeah.

Speaker B: That's discipline. Yeah.

Speaker A: It depends on who they're sharing it with. Right. So there's a lot of, there's a lot of projects, uh, that are going on between research, research hospital, academic research centers and pharmas where this is being debated right now. So, um, yes, there has to be a better mechanism because as you rightly noted, DE identified doesn't necessarily mean DE identified. You can break that out. So that is, that is, that is an area of concern, frankly, not one I'm prepared to answer because I don't have the answer.

Speaker B: Yeah, well, answer's gonna vary too, so no. Great. And thank you to everybody for joining us today. Please tune in as this show broadcasts weekdays at 2:30pm Eastern, 11:30am M. Pacific. As always, you can track me on LinkedIn @justintbarns or track me on Twitter at Hit Advisor and use the hashtag thisjustinradios and respond to your comments from the show if you missed any of this episode or want to hear more. All my shows are posted at YouTube, Spotify, Apple, iTunes, Amazon Music, Megaphone, SoundCloud, iHeartRadio and tune in and you check out the latest content and thought leadership we just posted@justinbarnes.com appreciate everybody's time. Thank you so much. Have a great rest of your day.

Speaker A: Sam.