The Three Words AI Won't Say: Inside Mednet's Bet on Honest Medicine

Speaker A: The worry is in these spaces where the knowledge doesn't exist. Right. Cause AI is never going to tell you. I don't know, like a good expert would tell you that, right?

Speaker B: Yeah, yeah. It's not designed to do that. It's always going to give you an answer. Confident answer, Very confident answer. And make you feel good about asking that great question that you asked. Exactly. Um, we actually, when we designed like Medna AI, like, we actually designed it to say, like, that the answer is not known. Like, not to extrapolate, not to. Because you'll see this. Like, you'll ask. You know, I had an experience recently where I was reviewing queries that failed RAM and the AI, and then I put it to another AI and the other AI gave a very confident answer. And then it went back and I had a reference and a resource and I went back and I looked at the reference and it was like this, like, really bad data.

Speaker A: I'm Dr. Amaravari.

Speaker C: And I'm Dr. Anthony Parafin Body. We are physician executives and we're the hosts of the Value Health Voices podcast.

Speaker A: We started this podcast to, uh, break down the business and policy forces shaping American healthcare today.

Speaker C: And the system is complex, driven by incentives, regulations, and payment models that aren't always easy to see.

Speaker A: On this show, we cut through that noise and focus on what actually matters, from Medicare reimbursement and PBMs to 340B and private equity.

Speaker C: We talk with the people shaping these decisions and we bring a perspective from both the front lines of patient care and the C suite.

Speaker A: Our goal is simple, give you clear, practical insights to help you lead in a system that's constantly changing.

Speaker C: And so if you're a physician executive, a, uh, hospital administrator, a benefits consultant, or a policy professional, this show is built for you.

Speaker A: Let's get into it.

Speaker C: Well, Amer, um, we're back together and this is going to be a fun episode, not only for me, but I think especially for you, as we have a guest tonight who is a very close friend of yours. And we are excited to talk about what is a kind of modern reality around AI and this abundance of data around us. It is a, uh, for clinicians, a paradox, a simultaneous abundance of data, a lot of information, but at the same time not enough useful information. And so we're excited to talk about that with your good friend, who I'll let you introduce.

Speaker A: Yeah, and exactly. And today we have, uh, Dr. Nadine Husri on, who is an associate, uh, professor adjunct at Yale School of Medicine, uh, the co founder of mednet, which we'll talk a lot about, uh, throughout this episode and what that is. Uh, but the reason I know her is we were, um, co residents back during our radiation, uh, oncology residency and study partners for those two years. Spent a lot of time together. So near and close to your friend. She's obviously highly accomplished, having received both the National Science foundation grant as well as an NCI grant and been featured on numerous, uh, media publications including New York Times, Stat News, and techcrunch. So welcome, Nadine.

Speaker B: Thank you so much for having me.

Speaker C: Yeah. Ah, this is great. Ah, we've done obviously the work that we always do to prepare for this episode and learn about all of your accomplishments and the way you have built a platform and a community where primarily physicians get together on MedNet to, uh, discuss what are some of the most vexing, complex problems for which there are little to no data and where their experience really provides, uh, together a kind of answer, a pathway forward to handle these tough, these tough clinical questions. Tell us, you know, you. This was years ago now that you founded MedNet, uh, not to age you or age any of us, but tell us a little bit about the core problem you were trying to solve as you started your platform.

Speaker B: Sure. So MedNet basically was founded to solve one problem and one problem at all, uh, for physicians, which is that we don't always know what to do for our patients. The data shows that physicians have questions with almost every patient encounter. About half that time, their questions just go completely unanswered. Right. And, um, you know, I want you to go back in your mind to like 2014 and this idea of having questions that, uh, you couldn't get answered. Even back then, before the age of AI and ChatGPT, it wasn't that outside of your life of medicine, it wasn't that common. Right. You would go online, you do a web search, you Google something, you could almost always find the answer for whatever it is you are looking for. But that just didn't happen in medicine. Oftentimes we were just kind of guessing the right thing for our patients. That's because the information is often something. You can't go and do a, uh, a PubMed search, I mean, sorry, a Google search or Even often a PubMed search or, or look it up in a textbook or up to date. A lot of the time the answers to really difficult questions are basically in the minds of other people, of experienced physicians, of experts in the field who've, you know, maybe seen something much more frequently than you, than you have. Um, um, and so that was, you know, what, what led to, um, to Mednet.

Speaker A: It's almost kind of like uh, how we used to do curbsides. Right. But it's, your platform is a way to almost have curbsides with experts around the entire country on, on different topics.

Speaker B: Yeah. And like in reality, the way that this whole thing started was I was curbsiding a few physicians. So I don't know if, I actually don't know if you know this. Amar. Many years ago, my dad was diagnosed with cancer and he's doing great now. But back then I did what everybody does when it's, when it's your own family member. I got on my computer, I sent some emails and I reached out to experts around the country basically just to ask them what I, we could do for my dad. And you know, thankfully he did end up getting really great care, um, and is doing well today. Um, but at that time I was very grateful that these experts not only replied to my emails, but actually got on the phone with me and answered my questions. And I, I wouldn't have thought much more about it because that is what we do all the time in medicine. We ask really smart people, you know, what, what would you do for your patient? I just happened to have an older brother whose background is in technology. And so my brother Samir started asking me questions about how physicians share knowledge with each other and more specifically how experts share their knowledge. And he was struck by two things. The first thing was how much expert knowledge is trapped in their heads. There's only so much that's, you know, published in the literature, that's, that's presented at, uh, conferences. A lot of the know how, the how to, how to apply data, what to do when there is no data, a lot of that's trapped in their heads. And the second thing that he pointed out is that when it is shared, it's shared in a very one on one way. You know, in this situation it was email, it was on the phone, you know, it could be text messages or group, you know, group chats. It's never really documented and shared in a way that helps other physicians who are not involved in that one on one conversation. So that was actually the true, like, impetus that, that led us to starting medinetics. Not something I ever would have even considered, to be honest at the time.

Speaker C: That's ah, so interesting. Yeah.

Speaker A: And when you started out, you, you focused on radiation oncology, but since then you've expanded to other disciplines, correct?

Speaker B: Yeah. So, um, you know, the first six Years we actually were only an oncology platform. And in my mind it's kind of like all I really cared about. Right. Like, you know, like you live and you breathe and you always think about cancer and it fascinates us and we're always like, thinking and ah, digging for answers. And I loved kind of getting the community together. What it was, whether it was radon, the med, oncs, you know, Pete's, the gynon, et cetera. And then Covid sort of, sort of made me change how I saw the world and made me realize, you know, how helpful men that would be outside of oncology. And that's when we actually started to expand. Um, so we're now in um, all the internal medicine specialties right now opening up to internal medicine, primary care. We have psych, we have neuro, we have derm, ah, offtho, ah, a couple of surgical specialties, neurosurgery, urology. So really starting to expand.

Speaker C: Is it right that one other part of your, that's uh, that founding period, that early period, let's say the first several years, five years of MedNet was also focused on using machine learning to match clinical questions with very specific clinical trials that may be applicable to your user's patients. Is that right?

Speaker B: Yeah. So very early on, um, we knew like, if you wanted to answer, the mission was let's answer every doc, every physician's question. And we knew there was two ways to do that. One was you could use machine learning. You could, you know, take a question, you can recognize what the question is asking and actually find the data that answers that question. And back then I didn't, we didn't have generative AI. So in my mind it was question, here's the data as opposed to like a generative AI that actually will, will like give you a narrative uh, of what the data says. That's. We knew it was like a machine learning problem. And then there was like the cultural problem which is like, how do you get people who are used to doing, you know, phone calls, email, talking in the hallway, texting each other, how do you get them to now actually start sharing that knowledge and having conversations online? So we started with a very difficult cultural problem, knowing that we would always go and focus on the more difficult technological. But the first time we actually started to do the research was around clinical trials. So that was the National Science foundation grant where we um, we actually worked with swog and we um, we were taking questions, breast cancer questions, and seeing if we could actually recognize um, which trials match those questions. Um, so that was our, that was actually our first, the first time we did anything around like clinical trials, around machine learning. And uh, yeah, it was a very, very limited data set. It was, there was very little that the models could do. Um, um, we had to train them, we had to abstract, manually, abstract a lot of data, standardize it all, you know, and like fast forward to today where it's kind of like the models can just do so much more. And obviously we've, you know, that idea that we had that we were sort of developing the models around back then, we've actually like completely built it out today.

Speaker A: Yeah.

Speaker B: And not just, you know, NCI trials, but across, you know, across all oncology.

Speaker A: I'm curious back to those beginning days because it was such a new and novel idea that you had. I mean people had like forums that were student doctor.net and everything, but nothing like this question answer type thing. Were these experts, were they reluctant or hesitant to share expert opinions? Which were they worried about liability or anything like that that you'd have to, with, with that culture you're describing that you had to surmount back then?

Speaker B: Um, they were not. And I think a large part of that is that we made the decision early on is that we weren't going to talk about spec. And we didn't do that to avoid liability. We did that because we wanted the answers to be as helpful to as many people as possible. And so, um, uh, that was basically modeled after Stack Overflow, which I don't know. You know, that's a platform, it's a question and answer platform. But there was like pretty tight moderation around, you know, what are the right questions that can, can be posted. So very early on we, we actually always had moderation, like a large amount of moderation around the questions. Obviously the answers, the experts are involved because the goal wasn't just chat, chit chat. The goal was help physicians get the best information to make great decisions for their patients.

Speaker C: Yeah. And um, thinking back also to the time when you were oncology only focused. Of course oncology is your clinical training and your, and your specialty. But I'm wondering if. So as part of researching this and reading things you've written, I've learned something that I didn't know and that is how what a small percentage of the recommendations and guidelines in our National Comprehensive Cancer Network, NCCN guidelines, uh, which are looked at as a gold standard, what a small percentage of those guidelines come from high level, the highest level evidence. Is that something that you realized years ago? And I'M all like a decade late realizing it myself. And that fed into why you said, okay, we need to solve this problem. Because, I mean, even in our guidelines, the, the quality is. I mean, there might be the best ones, but the quality is still relatively low.

Speaker B: Yeah, uh, uh, I mean, I remember reading the data a long time ago and not being surprised that the, the most of the, um, only like, I think it's like 7% of the recommendations are come from phase three randomized trials. And then like the category one, uh, data, and then it goes to like category two. And a lot of it's consensus, so I don't, I wasn't surprised. But the thing it was, it was really more like just the clinical practice then, like seeing patients day in and day out and, and knowing that you couldn't always, like, fit them. You know, NCCN's got these great decision trees, and they just don't fit in the decision tree. And, um, and that, that just comes from like, you know, being an oncologist and trying to. I remember reading a tweet back in the day many, many years ago that was like, anyone who thinks that AI can replace physicians has, have never tried to apply guidelines to your patients. And that's just the reality of oncology care, where, like, patients are just so. Everything's getting harder, you know, they're getting more complex, the clinical situations are more nuanced, and then sometimes because you have more data. We have more data than we've ever had before. We've had more, more phase three randomized data than we ever had before. It actually is harder to know what to do for your specific patient's clinical situation.

Speaker A: Yeah.

Speaker C: Uh, so, yeah.

Speaker A: And you've spoken about this, uh, about this doubling time of medical knowledge with all the amount of data going out there, because it used to be, I think, you know, years, and now it's shortened down to almost 73 days. Can you speak to that a little bit about. About how the MedNet's kind of helping navigate that?

Speaker B: Yeah, So I, you know, the first issue of, um, just trying to get, like, get read through the data and try to figure out what, what to do. I think most of us will just. Our first pass will be. Will go to the guidelines. Right. So we have something. It's not like we're, you know, flying without a parachute. Like, sorry, my idioms are not very good.

Speaker A: Oh, mine are, Mine are harbor. That's first generation. Right. We mix them all up.

Speaker B: But, um, you know, like, we have these great guidelines and, you know, as I'VE learned a lot more about like the um, the subtleties and the nuance of like how decision making happens across medicine. Like, we actually have amazing guidelines that are updated multiple times a year in uh, oncology, and that's rare for any other specialty, you know, maybe like cardiology. But for the most part, you know, a lot of guidelines are updated, you know, two, three, four, five, maybe 10 years later. So it is this cool thing that we've got, oncology, where we have up to date guidelines, but they still don't. Our patients don't always fit in them. And then often we go to a dual literature search, right, and we've got our retrospective Data, our Phase 2 data, our, you know, our other data that we can kind of try to figure out sort of, you know, what, what to do with. At the end of the day, all of that data though, you know, whether it's, you know, obviously there's high quality data, there's good data, there's less high quality data. At the end of the day, you're just like, well, what do I do? You know, like you're like, you're looking at it and you're like, I went to med school, I've done my residency, I've been in practice for this long and I'm a smart person looking at all this information. But I like, what are other smart people doing? You know, like, what are the experts doing? What are the people who have like way more. What are the people who are like running the clinical trials doing? The ones who are seeing, only seeing this type of patient day in and day out. What are people in the community doing? Right. The vast majority of patients are treating the community. And so there's this, you know, there's this knowledge base that like, we, we don't know like what's going on. And so that is kind of like where MedNet does come in, right? You might, and oftentimes you have an unanswer, you know, and like you think about like the last time you treated a patient and um, weren't sure what to do, you kind of like, you knew you wanted, you knew you wanted to go in one direction, but sometimes you just need the validation, right? You need, you want to know like what, what the experts are doing, what, what your colleagues are doing. Like are, are you crazy or does this make sense to other people as well? And oftentimes just that validation is really meaningful.

Speaker C: As a physician, you've given us a, a good background as to why users physicians would, would use MedNet might be illustrative if we talk about an example, maybe you could. Nadine, take us through the kind of difficult. Let's keep it oncologic. The kind of difficult oncologic question for which physicians would turn to MedNet.

Speaker B: Uh, you know, I was thinking recently about a question I posted. This is a while back. I had never treated tracheal cancer before. I don't know if either of you have treated tracheal cancer. It's rare twice, probably, but thoracic radiation, oncologist, you know, the patient, you know, that's. That's. That was. That was the consult that I got. Um, um, and I'm going through and I'm like, looking at what the. Going back to Perez. What's in the textbook? What, what. What's. What do the guidelines say? Do you treat the full length of the trachea? Do you just focus on. On where the. The tumor is and, you know, what do you do about the lymph nodes? Uh, I, uh, remember posting on. On Med. That and I, I like, kind of had an idea of what I wanted to do. I talked to my colleagues at Yale, and I got three answers from three different experts. And they were all different, completely different. And they all had data to back it up. But a lot of it was like, the clinical experience, maybe like how their mentors had taught them. They were rooted in, um, you know, the. The pathophysiology of, like, how these tumors tend to spread. And so it was validating that, like, one of them, the one that I felt more inclined to do that I thought was better for my patient because of the toxicity profile, you know, their performance status, their comorbidities. You know, seeing that another expert say that it was a reasonable option helped me decide what to do for my patient.

Speaker C: That's. It's a great example. And I can think of many instances where myself, I've asked questions or responded to questions on MedNet. And, uh, yeah, that's what you describe is a lot like my experience using it. You know, I was thinking, uh, again, as part of preparing to talk for you this evening about some very interesting predictions you made a few years ago now, in a period right before the world changed, not right before COVID where the world changed the first time, but right before the world changed because of the arrival of widely available large language models, in particular chatgpt. And so I would like, if you don't mind, to summarize what those three predictions were, and maybe then one by one, we could talk about how what you predict either came true, partially Came true. Where we stand today. Does that sound like a reasonable plan?

Speaker B: Yeah, it sounds okay. And then for context, this was the 2022 ASTRO annual meeting, the Presidential symposium.

Speaker C: Yes.

Speaker B: And it was entirely focused on the application of AI. And it was like the whole, you know, Amar had spoken at the Presidential symposium this, this past year, but there was three different sections and there was just experts in, in radon and AI and it was kind of like, you know, radiation oncologists have always been ahead of the game when it comes to, to artificial. I mean we've been talking, we've had sessions in AI and in our, in Astro since probably the 2010s. Yeah. Like going up back a long time ago, probably even before then. But then it kind of got into the more clinical application.

Speaker C: Yeah.

Speaker B: Um, um, so the Presidential symposium was focused on it. I, I was asked to speak and only because I had done that National Science foundation, uh, funded study, but that was my entire experience. I'm not a machine learning researcher. Definitely was like overwhelmed by like imposter syndrome of like what am I going to talk about? You know, and I didn't, I wasn't going to come and talk about the models that we did because it's like, you know, my um, colleague Sanjay ah. And Neha, he had, he had been doing like the, the actual like research and I had, doing kind of like the more you know, the abstraction and, and the clinical aspect. So I was like, what am I going to talk about? I'm an expert in knowledge sharing. So I ended up doing a talk and I probably. I've never spent so much time putting together a talk as I did with going back and looking at original resources. And I actually went back to the history of knowledge sharing in medicine, you know, and like when did we start documenting knowledge? When did, when did the medical textbooks come around? When did journals and like how did the evolution of knowledge sharing happen in medicine? And kind of took it all the way to the era of AI.

Speaker C: Yeah.

Speaker B: Um, so that was, that's sort of a background of what that talk was.

Speaker C: So to put, to put it.

Speaker A: So maybe. Sorry, yeah, maybe I'll say the predictions and then we'll go.

Speaker C: Hey, before you do that, Amar, I just want to put it another way. Is that for those who aren't in the radiation oncology community, I mean this is more like this is being a headliner. This is a big deal. This is the biggest radiation oncology meeting in the world. And the, the session in that meeting on which there's the most eyeballs and the most focus. So that's what Nadine. At the young age at the time, because I was 20, 22 and you just said you're for. You said you were 44, I think. So you're very, very young in your career to arrive at, at, at that moment. So amazing. So Amar, go, go ahead.

Speaker A: Very good points. Uh, Anthony. So the, the predictions that our, our, our Nostradamus here made is pretty much that AI systems would guide decision making where knowledge is known. Number two, for unknown knowledge, AI would help bring the collective knowledge and experience of our colleagues to the point of care. And three, the role of the physician would shift from an inquirer of information to an interpreter of knowledge and caregiver. So maybe we start with the first one, Nadine, and you can walk us through that. How, what was, what do you feel about that prediction that AI systems will guide decision making where knowledge is known?

Speaker B: That's actually, I feel like the easiest one. Right. Like, because we see it every day. If you've used ChatGPT or Claude or Gemini or the more uh, medical focused AIs, you know, docs GPT or open evidence or now we have Chat GPT for clinicians. You ask a question, it recognizes the question can actually go through, uh, you know the one, I think the, the clinical ones actually go through a limited data set and can actually write up. I know I'm saying it like it's stupid, but it's kind of amazing, right? Like, like an LLM can go and actually write a narrative, you know, uh, and it's all based on math, mathematical models, but like can actually give you a good answer and the right answer. And that's something now like that we do just every day. And um, that is sort of just like ingrained in our clinical practice. That didn't happen. It didn't exist three years ago.

Speaker A: Uh, with this prediction. Were you worried at the time or even now, I guess that with all these companies doing these investments in LLMs in this space, like you said, they're kind of going almost head to head a little bit against what you are providing at uh, MedNet. Right. So can you talk a little bit about what concerns you may have had and, and how you thought about them?

Speaker B: Yeah, I mean this idea of like collating the knowledge that's out there and making it more accessible to physicians. I mean the real, like the real pioneer, right, was actually Bud Rose back in the 90s when up to Date was developed as um, this textbook that was constantly being updated with the uh, you know, with the latest and the greatest information that's out there. So I never, I always felt like those resources right. Like compliment what MedNet offers because we really are in these nuanced, complicated, kind of like that decision that end of all the data like and then what, and then what's the decision that you make? Um so I never like at the time I knew we would do it and um, I even like thinking back before we, you know, developed MedNet AI just like it wasn't. It was like a given that there, that like a lot of people would do it.

Speaker C: Yeah. And going, going back. I mean you said that this first prediction was, it was the easy one. But you know, to remind the listener, I mean you, you.

Speaker B: Well no, but technologically very terrible.

Speaker C: Well no, no, I, I didn't even mean techno. I, I mean to say that your insight into it was actually not as, as trivial as you're making it out to be because this was before that this, this ubiquity of, of large language models. Right. It was uh, 2022. And. But this, this business about guiding decision making where knowledge exists is, gets back to the paradox that we started the episode with because there, when you get into uh, the more complex clinical scenarios for which physicians are most likely to turn to LLMs and ask these questions, the performance is, is actually quite poor. Again, in preparing the background information for, for talking to this evening found this data points about large language models performing in terms of complete completely accurate responses for difficult oncologic questions. Something like 30% or less. So most of the time being wrong and then uh, only about 25% or less providing any information that was clinically usable. So being perfectly right, 31% or less any information clinically usable for more complex questions a quarter or less of the time. So I mean that gets directly at uh, doesn't it? This business of information everywhere but maybe getting the wrong answers or hallucination as, as it's been been coined as a term for it.

Speaker B: It's funny, I gave the talk last week and the study that you mentioned, I like had it on a slide, but I didn't mention it was published in December of 25. I'm like it's outdated. We live in an era now where I was like the study was published in December, but like the models that they used were from you know, like the summer, you know, a year, almost a year ago. And should they even be citing this data that I think that regardless of the percentage. So I didn't, I kind of like had a slide And I was like, I'm not going to talk about this because who knows what, you know, with the models that we have today, what you would find. But regardless of that, we know that these models do really well on board examination qu. You know, board exam questions where the, there's one right answer, right. And, and the answer is known. Um, but then when you get to these clinical situations, you get to a nuanced patient, if, you know, you ask, how do I treat like a stage 3 lung non small cell lung cancer that's unresectable? You know, it's like a, it's a textbook answer. Right. But how do I treat a stage 3 non small cell lung cancer on a patient with a poor performance status who um, can't tolerate systemic therapy? Right. And now you're getting into like the complexity, the nuance. Um, so.

Speaker C: Or who can't tolerate this drug and they're on some other drug. Which means if we had to choose, well, if we chose drug A, that would combine with the drug that they're on for this other condition and then they would be in great difficulty. So we should choose drug B or C. Yeah.

Speaker B: What do you do for their, their, their adjuvant immunotherapy? They're on a steroid for their autoimmune disease. Like it's, you know, it's, it's uh, that's, that's this, that's just oncology. Like, it's, it's so complicated. I don't know. I, I did, you know, strictly did lung cancer, um, recently. And so it's every, I mean it was, I had few, I had few patients that really fit into the guidelines and were, you know, didn't have this level of complexity. But, but that's kind of the shortcoming, right? The what you get out of a large language model is only as good as the data that it uses. And the data is just not there. Um, and that's like the real problem.

Speaker A: And that goes exactly to what you, how you predicted it, that AI guides decision making where knowledge exists. The worry is in these spaces where the knowledge doesn't exist. Right. Because AI is never going to tell you. I don't know, like a good expert would tell you that, right?

Speaker B: Yeah, yeah. It's not designed to do that. It's always going to give you an answer, confident answer, very answer, and make you feel good about asking that great question that you asked. Exactly. Um, we actually, when we designed like Medna AI, like we actually designed it to say like that the answer is not known like not to extrapolate, not to. Because you'll see this like you'll ask. You know, I had an experience recently where I was reviewing queries that failed. Right. I meant the AI and one of them was kind of like unusual question but like it did, it failed. Right. It didn't give an answer. And I went back and I looked, I, I entered it in and I kind of saw what Medina and then I put into Jack GPT and actually ChatGPT, any given answer said this is really something that's unknown. And then I put it to another AI and the other AI gave a very confident answer. And then it went back and it had a reference and a resource and it went back and I looked at the reference and it was like this like really bad data. I mean it was based on like, like a very large database, like I think with billing data. And, and to make that assumption based on like pretty low level data, not that assumption, but to actually definitively say this is the answer, but the, the data that it's based on is not that strong. That's just how these systems work. You know, there's, they, they, they're not, they're, they kind of like, you know, for all intents and purposes they want to please you, they want to make you happy by, by giving them. It's a great, it's a great user experience to get a very definitive answer. It feels good when, when you get a definitive answer. And it felt less good to, to get so from the AI. They said this is something that's really not known.

Speaker C: Yeah, it's interesting how you're, you're setting up and ah, you have set up your platform to provide a level of honesty and in a way hard coding in a kind of humility that is a very desirable characteristic in humans and certainly in humans who have real responsibility. You know, that's, it's actually one of the most cited characteristics in CEO coaching and in what boards of directors in very high level. So Fortune 500 and certainly Fortune 100 firms say they want in a leader. And so that's just so interesting that you've done that and where the other AIs fail. What I want to do now is actually get to your second prediction and just let you go to town on it because I think this is really in the wheelhouse of everything you've been working on for a long time now. All right, so to remind the listeners, prediction number two, back in October 2022, one month before ChatGPT was for unknown knowledge, AI would help bring the collective knowledge and experience of our colleagues, so physicians to the point of care. All right, tell us about all that.

Speaker B: I mean, that essentially that's what MedNet is, right? It was. That's what it's been doing for over 10 years in medicine, which is focus in that area of unknown knowledge, document it, share, you know, share experience, expertise, survey the community. We, we always, you know, we do polls a lot on Med. Then, um, and you'll get these polls where it's like 20% is A and 32% is B and, you know, like 15, and it's. It's all over the place. That's. That's. That's medicine, right? That's like, we just don't know. And I didn't. Honestly, I made the prediction, but I, like, didn't know what it would look like. I didn't know it would look like generative AI. Like, I didn't know. I just knew that our goal was that we would be able to bring that wisdom and that knowledge and that clinical experience of the community to the point of care. And now it's like, you know, it's something that we built where we can see, like, you can ask a question, you get the data, you get the guidelines, and you get the community experience. That's giving you a very holistic sort of way of thinking of approaching a specific problem. The crazy thing is, like, when we. When people do queries in Medna AI, 80% of them actually result in. In Q and A from MedNet. So 80% of the time your question is pulling from the experience and the knowledge of the community, which I think is probably like. It's like, we could say like a standard of sort of like, when. When do we. When does it help? When, you know, the collective knowledge of the physician community, when should it be, like, informing our decisions? So. So that. That's. That's what. That's where we are today. That's what we. What we sort of, you know, launched, I think, maybe six weeks ago.

Speaker A: Wow.

Speaker C: Uh, I knew it was recent. I didn't know it was exactly so. So recent.

Speaker B: And I think that, like, you know, I'm so, like, um. Because I'm so, like, I've been doing this for, you know, over 10 years, and I really. I noticed some things that people would tell me recently, and I was like, I love. Yes, I love how you said that. And the thing that, um, I've been hearing a lot is, wow. It brings together the. The art and the science of medicine in one place. And I'm Just like, yes, you know, all these words. I'm like, trying to explain what it does and what's mednet? And then at the end of the day, it's the art and the science of medicine, which, um, is what we practice every day.

Speaker A: Right. Because the art lies in those disagreements and the nuances and uh, what different patients preferences and all that type of stuff. Right. And so it's not just pure science that you see in the clinical trials and the data that's published. So you, you, you are bridging those two, uh, with this digital hallway that you've created.

Speaker B: I don't know if you ever heard the saying. It's like there's probably some Greek philosopher and I don't know who, who said it, but there was, uh, that medicine is the most. What is it? Medicine is the most scientific of the humanities and the most humane, uh, of the sciences. And I just, like, I love that. That just captures, like, you know, what we do every day as physicians where sci. It's. It's an art and a science. It's. It's always an art and a science.

Speaker C: Absolutely. And so in what you've done, and this is in some way, uh, your, Your value proposition, what makes you different is it's a way, uh, for AI in a clinical setting to preserve nuance, show disagreement, because there's value in that and be transparent about who's saying what and why, where. Where. Where the data are coming from that informing what people are saying. And, uh, and again, be honest about where an answer doesn't exist. Just what you want from a, A good med student or a good trainee. Right. I don't know. Don't pretend, you know this thing that could actually hurt the patient.

Speaker B: Yeah, for sure.

Speaker A: Maybe tell, uh, us a little about this last prediction of yours, which is, I think a little harder to probably just, uh, encapsulate so well. So maybe you could talk a little bit about that, which is the physician as the interpreter. And I forget how you. How else you phrased it.

Speaker C: Can I, Can I, you know, can I give it a shot?

Speaker B: Actually, the.

Speaker C: Cause I got your article right here.

Speaker A: Go ahead.

Speaker C: I wanted, I want you to. I want to get rid of it.

Speaker A: Yeah.

Speaker C: So the role of the physician will shift from acquirer of information to interpreter of knowledge and to being more of a caregiver.

Speaker A: That's it. Yes. Thank you.

Speaker B: So this is the, this was the aspect I was like, least comfortable with. I was like, trying to like, put it all together. I could totally spend lots of time working on this presentation and trying to figure out sort of like, what is the future of medicine? Because this AI revolution is happening. And I realized, so, you know, when you can go to an AI and a community and ask questions and learn, or let's take the community out, right? You just go straight to, like a straight AI. It's based on data. And if we're saying that what a physician does is just input the right questions, get an answer, treat a patient, and now they're just like, um, sitting at the bedside and holding hands with a patient. Right. That. I don't think that's like a future that we want for medicine. It's not.

Speaker C: We.

Speaker B: We do want more time with our patients. We do want to do the hand holding, like a hundred percent. Uh, why we all got into. Why we got into oncology specifically was. Is to have that relationship and be there for patients in really difficult times. But the other. But the reason many of us also went into medicine in general and then like oncology specifically, is because it's hard. Being a doctor is like, it's a hard thing to do. Right? It's the science is hard, the application of it is hard. The training is rigorous. The problem solving is really, really hard. And that's. But we let. But that's why we did it. I mean, that's a part of, like, the. A huge part of the satisfaction that comes out of being a physician is that, like, having difficult problems to solve and solving them, whether it's, you know, whatever way we solve them, whether we look for the right, you know, references or we go to our colleagues and. But like, and then treating a patient and then, you know, knowing that we did the best possible. And, you know, obviously there's nothing better than having a good outcome for your patient. You know, best tumor control, lowest, you know, least side effect profile. Yeah. Um, and so, you know, this. This idea of, like, AI coming in and giving these very quick answers. There's this fear. Now among very much discussive medical education circles, where are physicians? Like, are we de Skilling physicians, especially when it comes to trainees, are they learning the critical thinking skills that. That are so central to the practice of medicine, that are so important to a physician who's honing their craft? Uh, uh, as a physician. Yeah.

Speaker A: I talked to a physician at a meeting I was at last night. An older physician who's a little skeptical about all this AI stuff. We were talking about this whole concept of the ambient AI scribes.

Speaker C: Right.

Speaker A: Which, as we know, are great because it reduces documentation burden, reduces physician burnout but his worries precisely around this that if, if you're not thinking through what you're documenting, are you. Are you actually still having that cognitive process occurring or, or, or are you missing out on that? And could it be a detriment to the patient? Potentially, yeah.

Speaker B: I think it's. It's really like, where, where do you do your problem solving and your critical thinking? And I'm, I'm that physician where writing it out is when I do my critical thinking, where I'm like, you know, you know, writing out the assessment plan. And I was taught this for one of my mentors in med school that was like, he would actually write his thought process out and you could read his note and know exactly what he was thinking. And I learned how to write notes from him, and that's how I would think is like writing the note. That being said, you probably don't need a lot of critical thinking in, like, writing, you know, about CT scan results or lab results or the, you know, the family history or. But the assessment and plan for me. But not everyone's like that. Right. Some people, like, they just. Writing is not when they do their thinking. Yeah.

Speaker A: And then. And some people, honestly, if you look at physician notes, let's look at the va. It's just a copy and paste mess. Right. And so there's actually more likely to be errors when people are doing that than probably using, uh, these ambient AI scribes. And so, you know, it's an, it's an interesting way to think about it because that's exactly. The counterargument I made to that physician was that, well, you know, people have been doing this for years. It's just copy and paste, you know, and so this is actually better than that. But if, if, like you said, Nadine, if, if your critical thinking is occurring during the documentation phase, and if you're a new trainee who's in that position, that could be a major problem.

Speaker B: Yeah, I remember, uh, you probably. Have you read notes. Have you gone to the doctor and read their notes about, like, your visit?

Speaker A: Oh, no, I would never.

Speaker B: Completely wrong. And I'm like, is this person listening to me at all? Like, that's not even. That was like HPI was completely wrong. Everything was. And then recently, I think it was probably the first note that I saw that was. It was so perfect. It captured, like, everything, like, exactly what I said, 2T. And I'm like, there's no way this physician wrote this note had to be like a scribe, an ambient scribe, because it was like a perfect, perfectly Captured the uh, the encounter. And I, you know, I think that's the benefit is like, you know, the accuracy there, it's, there's stuff, I mean that's the other thing is like we used to always, um, one of the things that people were spending a lot of time on was using machine learning to understand uh, medical records. And then because you know, like we said, there's only so much that's actually that we're learning from face to randomized data. So what if you go into the medical records and, and now you're using all of this data and medical records to, to do like scientific research, but garbage in, garbage out and like we just. So much of that data, so much of that information is wrong. And you'll, you know, I remember talking to people who were using these huge data sets and I'm like, well, how is it? What, what's it like? And they're like, it doesn't make any sense. You know what's coming out of it. Um, because the data wasn't good. So I think there is potential now that we, we are, but now it's like moving forward that we are going to be collecting better data in the patient record.

Speaker C: You know, one of the um, things that will hopefully, uh, and I think you've written about this, help us minimize some of these downsides is, is what you're doing with mednet and what you've been working on, your career. So scaling clinical wisdom to really help upscale. Upscale. Excuse me, upscale physicians and prevent um, you know, the descaling and everything we were just talking about. I wonder if a part of that too is training, uh, physicians really clinical systems, honestly, because that means, you know, the nurse and, and you know, PAs and et cetera, to be able to have efficient routines but also maintain innovative problem solving. And I believe this when this article we were just talking about a minute ago with all these terms, the deskaling, et cetera, um, no skilling or never scaling calls this adaptive practice. Is that, are you thinking about that mission in terms of how you're evolving MedNet at this time? Is that impacting your design?

Speaker B: I think, you know what happens is like we have this way that we're always practicing, you know, I see I'm a thoracic, so I'm like it's always gonna be a lung cancer example. I see a patient inoperable peripheral non small cell lung cancer. I, you know, 54 grade 3 fractions VMAT. It's kind of like the things that you see and you just know what to do right away. So you've got, like, your routine practice, and then you've got this break where a patient comes in and they don't fit like the cookbook, right? They don't fit like that. Routine practice. And now you do like that. Adaptive. Okay, well, let's figure out how to solve this problem. This patient doesn't fit into the normal way that you would treat like, or the, uh, you know, the guidelines way that you would treat this patient. And that's the opportunity where AI can actually be really helpful. Right? You're going now. You're searching for information. You're getting information from the literature. Maybe your. Your, you know, papers that you've never read before are being surfaced, right, that are actually a pretty good quality, um, data. But also you're getting. You're getting this, you know, at least on MedNet, you're getting, like, this expertise and this experience from the community at the same time. And that part is not just like, here's the data. That part is like, here's the problem. Here's how I think through it. And. And that's like the. That's, to me, like, my favorite thing about, you know, experts contributing on, um, medved. They're not just saying, this is what I. This is what you should do. It's. That's rarely an answer. The answer is often like, here's my. Here's my clinical reasoning and how I get to that conclusion. Oftentimes the conclusion is, I don't know. Right. You know, an expert will be like, I've never seen this before, but this is how I would think about it. And this is why these are different options that you could do. And so, as opposed to this, I think, like de Skilling, you're upskilling because now you're learning not just, you know, going back to residency and fellowship. You're always learning about how your attendings would think, but now you're continuing to learn from how experts think, you know, well into your, you know, your clinical practice, you know, and then that's, you know, expansive way beyond your. Your residency, your fellowship program, et cetera. So that's how I see this. This very human component. The. The expertise, the experience, the community helping us, like, get better at what we do as physicians, kind of, you know, everyone kind of talking about their critical thinking and their reasoning behind what they do, with what. Why they do what they do. Um, um. And that's how we, like, that's how we master that craft, right? That's how we develop that clinical judgment and get really good at as clinicians.

Speaker A: Yeah. And this concept that you're describing of upskilling and uh, you know, you've uh, said it's pretty much scaling clinical wisdom and you've also talked about this back in 2022, uh, that ultimately it's better if these AI tools are designed by us, not just for us. Right. So I guess I'm curious, what does that mean for the MedNet going forward? And ultimately how are you going to regulate what the AI does as, you know, as time goes on? Uh, at uh, MedNet AI?

Speaker B: Yeah, it was designed. So we have a team of. Right now we're six physicians at MedNet. So it's. Everything that we develop is product engineering, medical. It's those. Product engineering, medical design. Right. And so it's all done, you know, with physicians every step of. No, actually that's not how it should work. It should work like this. That's not actually what helps physicians. This, this is what. That's not actually what's going to get to the right answer. That's, that's best for a patient. It has to be this way. So that's how we're continuing. I mean it starts Obviously, like the MedNet M was co founded by physicians. So it started out with the mindset of like, what's actually gonna help my, what's gonna help me. Right. And what's gonna help my colleagues. And it continues now, you know, almost, almost 12 years, you know, after the, the company was first started. Our, our first actually, you know, our company values, Our first value is doctors first and that, that kind of shines through in everything that we do. That's.

Speaker C: Yeah, that's very impressive. And before we move towards perhaps a uh, closing or summary of the things we've talked about, I'm just curious. So the value to the physician user. We spent a lot of time talking about that. It's very clear why physicians would use MedNet. MedNet AI. But talk to us a little bit about the business model that you're using. So the value proposition, who's paying. We talked a little bit before about the seductive nature of these confident answers that other platforms give. You're providing this much more honest approach, this clinical wisdom infrastructure. So talk to us about, about the business.

Speaker B: Yeah. Uh, so there was no business when we started MedNet. Right. We're like, let's build something that it's gonna help physicians and then we'll figure it out later.

Speaker A: I remember you, I remember you saying it to me in our little office and I was like, yeah, that sounds cool.

Speaker B: It's just me and my brother. We'll figure it out.

Speaker A: Yeah.

Speaker B: You know, and then you know, we started to get the grants and um, you know, we went through Y Combinator and had funding from, from them and you know, raise a seed round and uh, the, the challenge in the business is there's no perfect revenue, there's no perfect model because a subscription is going to basically create barriers. Right. A ad model is going to like, you know, potentially could create bias. And who likes ads anyway? Um, like how do we do what we want to do without introducing bias? By keeping this like a physician only platform where there's this community and this trust and so the actual, the first business model that, that we uh, that I really wanted to go down the go in the direction of was actually the clinical trial model where like when we introduce clinical trials, can we also, you know, can we provide more information on clinical trials from study sponsors? Can we feature certain trials from study sponsors? That's still something that I think there's a lot of opportunity around that where you're raising. I mean the whole idea behind the clinical trials project and the grant is that you would raise awareness to clinical trials. When physicians are not even thinking about trials. That's a huge problem in adult oncology. It's very different from pediatric oncology when the first thing a pediatric uh, the first question a pediatric oncologist asks themselves when they see a new patient is what trial do I have for this patient? That's the first question a pediatric oncologist asks. That's like maybe first question adult oncologist asks is what's first line, what's second line, what's third line? And then when you have no other options, that's when we start asking ourselves what are the clinical trials that we have available for our patients. At that point they're probably not even eligible for for most trials. So, so this idea of like, you know, introducing clinical trials as an answer, raising awareness to clinical trials and then working with the study sponsors where we could, you know, surface more information about the trials, whether it's you know, the slots, uh, the sites, you know, the protocols, things like that, that's still like uh, you know, I think a uh, really great model that we're still working on. We run journal clubs. We've been doing that for many years on Medna. We originally developed the idea with ASCO when we did these journal clubs in from jco and, and then you know, we just happened to find out that the pharma was like really Interested in like journal clubs? And we're like, okay, well do you want to, you sponsored the trial, do you want to sponsor the journal club? And so, um, we, you know, we often can get sponsorship for, for journal clubs and then, you know, we just run it the way we do. They don't have any influence on the questions, the answers, the experts, we just do it the way you do. But then they kind of learn, you know, about the questions that are coming up in the community. I mean it's all anonymized, but like they get more understanding of sort of how are these trials, how is the paper actually being implemented in real world clinical practice, there only has to be uh, a few ways that the business can support itself and you don't have to like sub and then subsidizes everything else that we do.

Speaker A: Yeah, actually that, that brings up a question I think you probably want to address anyway also around data privacy because that's obviously something that uh, people are concerned about as time as, you know, as, as they join more and more of these kind of platforms with AI and everything. Can uh, you talk a little about, about how you protect data privacy?

Speaker B: Yeah, I mean for the community, we just, we've always not allowed patient level information. Again, not. It. It was just like the way that we thought it'd be the most helpful to most physicians. That was sort of like the single mindset behind that. But that kind of took care of the data privacy as well, you know, and, and with the AI feature, people do enter patient information. We could actually describe that before we store any of it on, on a database. So though.

Speaker A: And for the users.

Speaker B: Oh, their own data.

Speaker A: Yeah, yeah, yeah, yeah.

Speaker B: We don't really have, we don't have like user data.

Speaker C: I mean so not like NPI numbers or other license numbers, nothing like that?

Speaker B: Oh no, we do have. Yeah, I mean that's how we do a lot of verification. So we do have NPIs, which are,

Speaker C: you know, I suppose you could, you can Google anybody. You can.

Speaker A: Yeah, it's the first thing in the.

Speaker C: Well, so on a, on an individual level basis you can, I suppose. But yes, I guess it's more. Becomes an interest, an interesting problem. If it's a huge number like a Data set of NPIs and then that's used to, for unwanted like marketing purposes or some such thing.

Speaker B: Uh, as a whole data set don't have that level. On Medinet you have to say who you are. So there's no anonymity. Who you are, where you work. It's all like You've got a profile, your pictures up there. Uh, and again, it was designed that way where.

Speaker C: Yeah.

Speaker B: Who says something is often as important as what's being said, you know, and, and if someone's coming into a professional environment, like their name, their institution, you act also, you act differently. Right.

Speaker C: Level professional mechanism. Exactly.

Speaker B: And then like, you've got, if you're saying something, you're. And, uh, and it's not some anonymous name, like you've got some steak, you know, you got. What do they say?

Speaker A: Yes, Skin. Skin in the game.

Speaker B: Skin in the game. Right. Like, this is like, you're so. So that. That is like, not. There's no anonymity in that. I mean, there's anonymity in like voting on the polls. You can ask anonymously, but in terms of like, who the community is. Like, you. You see who your community is. You see who your call, who you're talking to and who your colleagues are.

Speaker C: Yeah.

Speaker B: Uh, a lot different than, you know, like a platform like Reddit or something, where you're like, is this person even a doctor? Like, yeah, right. Is this a bot?

Speaker C: So, uh, um, so in, in closing, I think it might be most interesting and practical to do two things. And the interesting one is if you can tell us, you know, what's next for MedNet, where's it going? And then practically we want to make sure people listening to us who don't, aren't already users know how to find, you know, where to go to be part of the community.

Speaker B: Right. In terms of what's next, we're continuing to grow across all of medicine. So includes, like I said, we're actually all of adult medicine and continue to grow in, in the, in the specialties as well. Future, you know, more surgical, more peds, more. More gyne specialties as well. The. It's funny, I used to give this talk and it'd be like, in five to 10 years we're going to be Google for medicine. And it's like now I'm like, well, you know, now we're there. Now we're like, you can come and you could search and always get an answer to your question. And it was like this fantasy I had that that now is, has come, has come to, to uh, become a reality. I. There's going to be a lot more about. Around. Clinical trials are still in year two of our NCI grant, so you can see a lot more about clinical trials recommended to you, not just when you're coming in and searching for them, but also when you're reading Q and A, you know, and you find that there's actually a trial that's recommended because it's relevant to the Q and A. Or you're doing a medna AI search, and a trial is like, the answer. I've seen. I saw actually, this perfect example where somebody asked. It was actually a community question. Someone asked a question, and the. The LM had actually, um, matched a trial that was literally exactly the answer to, like, the answer is like, put the patient on the trial and then, you know, you see where the trial is and it's like, you know, because you enter, you have. You have your zip code information. So it's like. But the trial is like, two miles from you and it answers your question. Like, this is pretty much the answer to your question, is put the patient on a clinical trial. Um, so there's gonna be a lot more of that as well.

Speaker C: Amazing.

Speaker B: Um. Um. Yeah.

Speaker A: And your platform, it's. It's, uh, just. Oh, yeah, somednet.org or.

Speaker B: Yeah, the mednet.org we have a mobile app, too. Um. Um, we actually just relaunched in the App Store. Um, so Android and iOS.

Speaker A: Thank you, Nadine, for coming on. This was really super informative and so excited to see where the mednet goes. Obviously, we both are big users, so, uh, very excited and thankful for having you on.

Speaker B: Thank you, Amar. And I'm a big fan. I'm a big fan of your work. I actually, you know, I think that it's funny sitting in that office back in residency, and I was like, this guy, he's like, he's too big for this office.

Speaker A: That's very kind to you.

Speaker B: We're going to be like, one of those. I feel like he's gonna be one of those. I would think, like, you'd be one of those people, like those speakers at Astro. At Astro. And it's. So now you're like, well, you.

Speaker A: You beat me to it, lady.

Speaker C: That's not funny. Love it. Thanks so much for joining us.

Speaker B: Sold this to make our chair proud, so.

Speaker A: Yeah, exactly.

Speaker B: Just make dad proud of us.

Speaker A: All right.