AI in Healthcare: Shadow AI, Trust & the Human’s Role - with Dr Keith Grimes (#70)

Technology's achieved escape velocity, as in healthcare will never capture it. It's escaped, right? But that's okay, maybe, because you know, as long as you use it usefully. Welcome to the Healthusiasm Podcast, a panel discussion on innovations in health and self-care.

My name is Christophe Jauquet, a health business expert, author, and professional keynote speaker. And every month or so I discuss with a panel of experts with positive changes that are shaping our health and happiness. And for this discussion, we have one of your beloved experts in the panel from Scotland but living near Brighton, medical expert in digital innovation. Welcome, Dr.

Keith Grimes. Fit like Arby and Go Scotland World Cup. Yeah, it's a World Cup. Are you watching football?

A little bit. Like I prefer rugby, but when the World Cup comes on, of course I get excited. So yes, I'm watching the occasional game, but it's always fun. Yeah, occasionally as well.

I'm a basketball fan, so I'm way into basketball, not too much into football to be. Well, the Knicks did something good with basketball, of course. Interesting. It was a very interesting final, and it's interesting that New York has won after 53 years or something.

So the Mecca of basketball backed the Mecca of basketball. That's basically what happened. But before we get into their conversation, shout out to our other panelists who couldn't make it. Aline Noizet, Estefanía 'Nia' Escobar-Kölle, and Mo Zouina.

We are looking forward to talking to you again. yeah, we almost saw each other just this week at Health. but I couldn't make it. I wanted to be there on Monday, but most of the events was privates, was VIP.

Not as much people as I would have loved to have seen. So I skipped this year's edition, couldn't make it. But yeah, tell me, you were there, right? So how was health?

Yeah, I wasn't. Did I miss out on something? Yeah, so as we're recording, it's Friday, and I went out on Tuesday and came back late last night. the reason I didn't go on Monday is it was my wife and I, it was our 25th wedding anniversary on the Monday.

Quite correctly chose wife over health. If you see what I mean, H L T H. But yeah, so I was there from Tuesday till Thursday. I've been to health a few times with the US and the you know, the evolving and improving HLTH Europe.

I and when I say improving, it started out great, but you know, year on year they've made like changes, some visible, some less visible, that meant that this year felt I really enjoyed it, had some fantastic conversations, learnt a great deal. But the general feeling that I came away with is that this year there was a kind of there was a sort of space to do that. And by that, they configured it, so there was seating space centrally and so on. There were some changes to how it was set up.

It just meant that hasn't that always been the case? Yeah, but previously there's been plenty of seats, but they were sort of scattered around, if you see what I mean. But this time they kind of centralized it. The heart of the exhibition, if you imagine, there's a cross and sort of center, big central area, and it just felt as though it worked and flowed very well.

Everyone you never had any challenge to either speak to someone at a stand, and my company Curistica had a stand with the Department for Business and Trade. So you could speak there or move around. There was always a way in which the conversation could be light and in space, or if you wanted to progress it, there was always a place to go. And health is if yeah, is famous for taking care of the attendees, and it was the same this time around.

The evening events were lovely, yeah. And then the Thursday is always the sort of barbecue down on the beach, which is next to one to come out. So, yeah, and Amsterdam was very generous with its weather, it was lovely, so and all the England fans disappeared for at least a little while on Wednesday night to go and cheer on their team. Yeah, I saw I'm in the WhatsApp group with all the Belgians present as well, and they were texting as well to be able to see the game together, which gives this nice kind of hype, even if you're not into football.

I mean, it's nice to have people in a different kind of mirror neurons pick up the excitement of those around you. Yeah, but it's good to hear, and indeed, I mean, the one thing that I've always said to the people working at Health is that what sets them really apart is indeed the way that you can meet people, the way that they organize their events. I mean, it's always very, you know, meet-friendly, not meet as in the food, but meeting people. and so that's that's one of the things that I always like about health.

It is set up, you can listen to the panels, you can listen to the keynotes if there's any, but at the same time, you can always meet up with people. You don't need to wait until the panels are finished. I mean, you just walk around and you go to the booths and you meet the people, and people are just and everybody's there. Was everybody there as well?

I mean, like the people you expect to be there. Yeah, yeah, yeah. There was I mean, you and I have both been in this space for a while, so it's also an opportunity to catch up with friends old and new. The UK contingent were there in very large force.

Every year, the Department for Business Trade sort of hosts like a group photo, and this year it was like absurd. I mean, there must have been a couple of hundred people just crammed in this space, so that was good. And I and I agree, I think what health understands very well is something simple, which is you take care of those little things that prevent you from fully engaging. So, like simple stuff, like when I came in, I'd come straight from the airport, but the cloakroom was easy to access and free to use, and you know, the check-in bit was all absolutely fine.

The food, the drink, there's always food, drink, coffee, water, nibbles. You know, you don't have to sort of go, oh, now I have to go and do X or Y. And when all those little annoyances from other places go, you can really be in the moment. And again with the talks, they'd slightly moved around where the talking spaces were, which meant the sound kind of stayed where it was meant to.

And I went to a few talks, and I have a sense this time round that the kind of the talk and then the engagement immediately after the talk, you know, the speakers are always very good at meeting folk, but that there was a real buzz. They had great talks, but then the conversations afterwards, it just it just flowed very nicely. And you were there at Curistica, right? Yeah, yeah, yeah, yeah.

So you were there for business, but just for the people who are not familiar with Curistica, what is it exactly that you do? Yeah. So I mean you're very kind, and you know, I'm a doctor by background, a general practitioner in the UK, but I was always a real geek and techie person. my company, Curistica, is a company that provides safety services, you know, clinical risk management and safety services for people that build AI or digital health technologies or deploy them, you know, hospitals, practices, and so on.

At its heart, I want to see technology that does great things in front of patients and the staff that serve them and their families, but I'm a doctor, it has to be safe. And whilst everyone wants stuff to be safe, it's remarkably difficult to know how to make something safe. And that's what we do. and particularly with new technologies.

So as a standard in the UK that's quite specific to the UK, which is required, and we do that work, it's called DCB. but it's a version of clinical risk management and AI governance. And when I talk to people about safety, we already deal with the safety of doctors and nurses. We know how to do that.

We know about drugs, everyone knows the part they have to play, even physical devices and some things. But digital and AI, people think about it differently. They sort of think this is someone else's problem to make safe, and then I can just use it. that's what we have to change.

But people need guidance on how, and that's basically what we do. To me, that would be the boring stuff, but well let me tell you, my friend, it is the most interesting bit. But no, you're not wrong about it. It's good that we share that we share responsibilities, then yeah, yeah.

I mean, like, you know, we all have different tastes, and yeah, on the face of it, I think people sort of go, no, I want to get the cool stuff done. And these safety people are there to just like spoil the party. Exactly. Actually, I think they're fair to say that traditionally that's been the case.

I think, you know, but everyone has a sort of part to play in that, which is that if people don't know the part they play in making something safe, and they think it's someone else's someone else's job, then yeah, they will feel that it's getting in the way. But my experience over the years, particularly when I worked at Babylon and I was a doctor working there with the AI teams and doing a bunch of things, when you bring it together, if you kind of correct that, when the safety people know more about what the product and the commercial and everyone are doing, but also vice versa, if you can think a little bit at the start about this, everything is safer, everything is faster.

and when it comes to like solving how to make really advanced and weird technology safe is really complicated, right? But that also means it is utterly delicious to think about how to do it. Like, like I said, yeah, and so I confess that you have to be a certain kind of person to really geek out on geek out on it. I'm one of those people, and there aren't many.

So, you know, we're that part of the team, you know? Yeah, well, the I know, I mean, and it's and it's probably the most valuable thing to do right now, but it's I the reason why I say it is that you know, I when I'm on stage, my role is to you know get people enthusiastic, health thusiastic, as I like to say, about the potential of AI and that they need to start about you know with it, that they that they could do this, that they need to think about that, that the world will change in this way.

And then I build scenarios about how the world will evolve. And then at the end of the keynote, where everybody's like, wow, this is amazing, and I love it. And then there's always like this one party pooper raising his hands at the end for the QA and then say it's like, yeah, but is it safe? And this is like the question that I least like.

I mean, they can challenge me on everything else because the answer I always give is that you know it's probably not safe today. I'm pretty sure it will be, but I don't care. And the reason why I do care, of course, but the reason why I say I don't care is because there's so many discussions and so many people involved in making sure that it is safe, that I think this is not my role. So I try to you know put it aside as quickly as possible.

But I do believe that it is very important. The one thing that I'm always afraid of, and this I think you alluded to it as well, is that if you focus too much on the safety or the safety of it, the safeness of it, it's it risks slowing things down that much that you get into that sphere where people just go into look for other solutions anyway, that maybe are less safe. I mean, the shadow AI, the shadow technology that we're always talking about, you know, people using WhatsApp to send some page director to somebody else.

Yeah, yeah. And so one of the things that I'm always holding people that working in safety, AI safety responsible for, and maybe even you know putting the blame on them is that you make things too slow. You make sure that people, because things are going so fast, definitely in AI right now. So the challenge for you and your team would probably be making sure that things don't slow down, that people just don't go into the shadow AI type of work.

Yeah, I mean, these are look, everything you just said there is completely fair and actually correct, particularly in terms of how people have approached things. But, you know, the first thing to say is that risk is everywhere. Like, and we live with it and it's okay. Like the practice of medicine is risky.

A doctor seeing you is risky, taking a drug is risky. Driving a car is risky. All these things are risky, but we do them every day and we do them mostly safely. And there's a few reasons for that.

Number one is some of the work that the safety people do, you know, that happens kind of behind the scenes and so on. And we know that there have been times in our history where we haven't thought about it, and then something horrific happens, and then things change, and then it gets wrapped into normal things. The classic one we talk about always is thalidomide, you know, the drug that came out in the sort of 60s and early 70s for nausea was had a great effect in terms of actually its therapeutic effect, but tragically led to you know terrible birth defects for around about 20,000 individuals and led to a massive change in how we deal with drugs.

And we've seen that time again in surgical practice, medical practice, breast implants, mesh implants, and so on. And something happens and then we go, oh, that must never happen again, and it kind of swings back and so on. You have to try and navigate-regulate things, and then we have to navigate a tricky line, but maybe I'll put it another way as like, you know, before people went to America, you know, America was discovered, you know, no expedition. There was the idea that there might be something exciting and useful and brilliant on the other side of it.

But I would say that you wouldn't then say, that's so brilliant, I'm just gonna swim, right? Because of course you wouldn't, like the risk of get, yeah, the likelihood of getting there would be very low, and the risks of trying to swim to America would be crazy. So people have a sort of baseline understanding of how you can approach this. But then there's dealing with uncertainty as well.

And so I would suggest that when you're heading for this big goal, there are certain things that you can do. If you spend a moment thinking beforehand and say, well, how can I keep an eye out for when dangerous things happen and how can I like proportionately prepare myself for it? And that's essentially what happens. And that's essentially what we're doing, is we're saying, right, you've built this cool thing, like I'm excited too.

I want the jetpacks on the moon, I want chatbots that people can use to give them excellent advice and everything like that. But of course, come from a medical background, I don't want anyone to be harmed unnecessarily along the way. So, what are the reasonable things that I can do to reduce the likelihood of that? And that is essentially what it is.

It's taking a breath beforehand and saying, okay, let's have a we think. What can we do? But I hope that what we do at Curistica, and there are other people out there, which is like, but we're gonna work our asses off to work out how to do it. We're not here to say no, we're here to say, yes, this is how you get there.

And that's part of it. The second part is like, once it's done once, like the world is weird, the world changes, you know, these are new things. Let's keep an eye on things. If something goes wrong, if we can see it before it gets too bad, and if we can do something about it, then yes, let's try something that's a bit unusual.

That's okay. You know, if we bring people along with us and they understand that, that's okay. I think, and that's what I mean about people having a part to play in this. If you think that something's built and you can just do what you want, right?

It's a bit like saying, here's your Ferrari, like it's got an airbag, it's got seatbelts, but you're like, you know what, I'm gonna do I'm just gonna go for it, right? That's dangerous. You learn how to drive, and hopefully, if you're driving some fast car, you'll bear that in mind. But you need to have a modest understanding of like how to drive, and other people on the road need to know how to drive too.

And we're not doing enough of that. Yeah, that's one thing, but I think what I really like about the way that you approach things is what you just earlier said just earlier with the jetbacks on the moon. I think you are very interested with all the things that are happening and all the things that are possible. So you want to push things forward.

So that's why I would believe in you as a as my safety guy for AI, because you're not just focused on safety, which sometimes is the things that I see with you know, regulatory people, safety people, medical people, legal people, they're more like worried about what shouldn't be done, but they don't really dream about what could be done. Yeah, it's all like I think that is where you make a big difference, I think. In my experience with you guys. I think that's definitely true, and even I'm guilty of it sometimes.

It's like the how can I say no? Like you're just like, well, well, you know, well, wait a second, you've got to stop now, and everything stops. I'm you're right. I'm a real geek.

I love I use tech all the time. It's my life's passion, I enjoy it, it's my pastime, and so I'm always playing with the new tech. And I behave like a child in this one, is like, this is really cool. I can't wait for other people to see how cool it is.

I want other people to play with me with this stuff, right? But I'm also like, yeah, but like they're not gonna do it if they're not comfortable with it. Like, I could kind of convince them, but I don't want them to get into trouble. So, what do I have to do to make this okay for them to come and play with it, right?

You know, it's that's the kind of childlike view of it. And of course, I'm not a child there, I'm an older person, but if you take that kind of approach, I do want like agents and AGI partners, digital twins, jetpacks on the moon. I just don't want, I want to know practically how we do it. Like, that's great, jetpacks on the moon.

How are you building it? Where are we gonna get it there? When can I do it? Right?

Like, tell me how. And part of that is like, and I don't want to, I don't want to get my jetpack on the moon, press the button, and end up on Mars, you know, right? Then I can play with it. And so you really keep a close track of everything that's happening in AI, and we're definitely gonna talk about a lot of the stuff that we are seeing happening right now.

But I think one of the things that you do really well, or where you keep how do you say, your finger on the pulse about everything that's happening in AI, is the WhatsApp group that you're part of, which is the AI and the NHS, is that it? Yeah, AI and the NHS, yeah. So yeah, everyone's on lots of platforms, and over the time I've been on the Facebooks and the Twitters or X and everything like that. I've kind of drifted a little bit away from open social media and now more of a member of WhatsApp groups, or slightly larger but closed groups, because there's a few things, you know, conversation tends to be kind of less wild.

But you know, you can concentrate, you can concentrate on certain it's just a nice way of doing things. One of those groups is called AI and the NHS. Now it was set up by Paul Joshi and Dave Triska, two great doctors, very geeky doctors like me. and shortly after they set it up, I kind of joined as well.

So I'm one of the moderators and admins in that group. It has about, it has, as of this morning, 796 members, which for a closed and active WhatsApp group is actually pretty big. And that's only people from the NHS, or is it open through anybody else? No, in fact, you know, Gil Bash, who you know, FinPartners, you know, big in the health tech space, does a lot of talking about it.

I added him yesterday because I was talking about it and he wanted to participate. So a lot of people, a lot of doctors and nurses and so on from the NHS, of course, right? But there's a lot of manufacturers and industry people as well who are building products to sell into the NHS. There's a lot of the regulatory people, and then there's a lot of people that are in the kind of procurement and management of healthcare.

So cool. So it's a really varied group, and it's a place where there's a lot of enthusiasts, so a lot of topical discussions happen. And it's quite a robust group. Now, when I say that, people hold a lot of different opinions.

It but it's actually what's nice about it, and it's always tricky, is that people have strong opinions and they'll challenge each other, but it's it generally stays on the right side of being respectful and so on. And that means that there's a kind of large group of people that will have active discussion. There's a lot of people that just kind of observe and you know chip in every so often. And I think that has a really positive effect generally because we discuss difficult subjects, people get an idea of general, genuinely what the frontline feels and thinks and what's important to them, which then can translate out in a way that is.

The front line, what do you mean by the front line? The people that are the front line, these are people, you know, it's the nurses, the physios, the doctors, the people that are. Oh, yeah, the people that are actually in the having the actual problems on the day doing it. The other bit that it has is it has a lot of kind of super early adopters doing pretty advanced things on their own.

Like we've got people running large open models, building their own tech, experimenting with new ways of working. But that's very, very much on the edge. And again, generally, safety people start you know, their eyes start to swivel when they hear that. But doctors are very used to working in risk spaces and managing risk.

That's only possible if you have a good understanding of what you're working with. So tech enthusiasts and people with the expertise and the knowledge, I believe, can push that bit further because they understand where the cliff edges are, where the landmines are. And then this community is a place where people who have genuine expertise in things like cybersecurity, data protection, regulation, safety, who are also working in this space, are happy to sort of say this is all sounding great, double checking you've thought carefully about this area.

It's kind of quite collaborative. But I mean, 800 people almost for to go to who chat away about AI, I mean, must go all over the place. You must have like hundreds, you know, notifications all the time. You have to mute it anyway, I think.

But then when you come back, it's hard, you don't have any overview. I mean, discussions go all over the place. How do you manage that? I mean, it's well, I personally I just I post a lot.

I'm off on the top poster in all of it. So, but that's just me. I know that's not for everyone. We're averaging about 500 messages a week.

And so some people will say, Oh, I was away for a couple of weeks, it came back. There's like one and a half thousand messages I haven't read. And I don't expect them to read through it. So people sort of dip in and out.

But we realized this early on. And one of the things, as I you know, use AI a lot, is we early on adapted, we used like we built, or I built an agentic process that takes that this is with the consent of the group as well, it's important. To say, but the discussions are essentially processed and anonymized, and from that we generate a newsletter. But the purpose of this newsletter is to actually help those people who say, Good grief, there's a thousand messages, right?

So it's like a digest that comes out once a week, and it covers, it has a sort of analysis. This is AI generated, but we but it's reviewed as well and carefully prompted and structured. Topics that we've discussed, humorous moments, some stats about numbers, all with the people's names aren't in there, they're kind of anonymous, they have sort of descriptors. The links, because people are constantly sharing links, so they're all collated there as well, and then a daily breakdown too.

So those people that want to sit on the sidelines, dip in and out, and just want to understand what the hot topics are and what the takes were can do that too. and the links probably I imagine. The links probably, I imagine, that are shared about certain announcements or scientific studies or whatever. Yeah, yeah.

Because they because as you know with WhatsApp, because it's not threaded, something you're like, oh my god, that was interesting, I'll come back to it. And then it's a hundred messages behind, and you know you can't find it. So, you know, AI isn't perfect, and some things will be missed, some things will be sort of gently misrepresented. So you have to be careful on that side.

But it does a much better job of something that if I'm honest, I wouldn't do because I don't have the time. And it's become very helpful. Now we're over a year into it with all the Christmas specials and the Burns Night specials and all that kind of thing. We've actually got a really good timeline of how people's opinions have changed as well.

Oh yeah. So could you could you share some something about it? Yeah, well, I mean, the ambient voice technology is probably the best example because we started doing it. You mean describing, right?

Yeah, scribes, yeah. So for context, I think everyone knows what a scribe does, but you know, records what's in the room, uses AI to summarize it when a doctor or nurse is speaking to a patient, makes it easier so they can focus on the patient. You get some very high quality notes, but you have to check it and goes back in. Yeah.

They've been around for a little while. They've been very popular, very widely adopted. Easily adopted. It's probably the technology that's been adopted.

I can't think of another one. Yeah, I can't think of another one that people have been so keen to use. Like every technology that I've deployed in healthcare, and there have been many, has always been a bit of a push. This is the only one that people have been actively like, no, no, no, I'll do what I need to do.

I want that, you know. And they'll do they'll even do clinical safety to get it in, you know. But what happened is when they came out, is I'll say Wild West, it wasn't quite like that, but there was it felt quite free and easy. No one was entirely sure what kind of device they were, were they a device, how can you use them?

But they were being used a lot. The reason I say the last year's instructive is in the UK, the CCO, the chief clinical information officer, a guy called Alec Price Forbes, on June the 9th last year, so just before health last year, put out a letter to all the practices in England, because the UK is four nations, but England's the big chunk of it, saying, right, okay, you're all using these things. Let me be clear. He didn't say exactly this, but you know, if you do this, number one, you need a clinical safety officer, someone that manages the risk.

I'm one, and there's a few of them around, and you need to do this standard to check that the manufacturer's done it and you've deployed it okay. You have to get the data protection right, you have to make sure that it integrates with the system right. it should be a device. If you don't do any of these things, the liability for this, my friend, is all on you.

And it was really clear. That point changed a lot because up until then, people were thinking, oh, it's not that people didn't know, but all of a sudden someone was saying, I'm telling you what you need to do, and if you don't do this, it's all on you, right? And so our newsletter started coming out just after that, and we often discuss ambient voice technology. So at the start, it was making sense of it and discussion about like, you know, it's great, and other people saying it's terrible.

It's become kind of much more nuanced over the year, and so now people are sort of talking about what they found after, you know, actually sometimes, you know, over a year, maybe a year and a half of use of it. They're picking up things that you couldn't have thought up beforehand about strange ways in which it changes how you practice, strange failure modes, opportunities as new things come in, they get discussed. And so you can see a much better critical appreciation. Like people are still very positive about it.

Some people are still quite negative about downstream consequences, but you can see in front of your eyes people working through collaboratively the things that you were uncertain about, the things that you're finding are very good. And then just see the progress in the way that they are using it, basically. Yeah, yeah. And it I think it's hard, it would be hard to pick up that kind of evolution without having gone through what we've gone through.

Yeah. Did you did you create a I mean you should make a make a report on that and just you know, the evolution, the perception of scribes in the past year? Yes, I mean, yeah. And this is something maybe you and I can do, sir.

And I mean, it's important for me to say that you know, this is a community of people, and you know, it's always, I hope, done with their blessing and consent, you know, and interest too, for the right reasons. And you know, I'm very aware that I'm a doctor, I don't practice anymore, and I am now in industry, and I have a business that does, you know, the direct interest in all of this for me too. So I have to always be aware of that. I hope, I mean, I feel that I'm doing this for the right reasons.

I hope that people understand and believe that. And so it's very important for me and the admins to appropriately care for that as best we can. It's a tricky space. With that, and if it were found useful to that group, then yeah, I think it would be very interesting, not only for the users of it, but also policymakers and industry and patients themselves, you know.

What do the people that use this actually think about it? I'm happy to go to the patients now because I mean one of the things that I always work on is the other side of the spectrum of your spectrum, which is more the consumer-driven health, where we see that more and more people are actively involved in their health, looking for solutions and so forth. And so also them, also these people, everyday people, they're increasingly using AI for their health. Even the ones that are not actively, I would say, or maybe not the biggest earlier earliest of innovators, let's say, they're starting to use health everywhere.

We saw about a year ago, I think Microsoft Copilot, one in five people were asking it health questions. It came out as a co-pilot health now. And there's 230 million questions per week on ChatGPT on health one year ago already, so it probably doubled in the meantime. So we see more and more people using health or using LLMs, ChatGPT, whatsoever what have you, to for their own health.

And then they go to the doctor. And we talked about safety, we talked about you know not slowing down things too much. What we see is that patients, consumers are not slowing down things themselves. No, no, they are using it in ways that maybe is not safe at all, but they are using it in ways, and that's part of the shadow technology, maybe that we are seeing outside of healthcare.

So, how do we deal with that? How do you I mean, how can we make sure that you know physicians do not treat it the same way that as they did with the beginning of Dr. Google, where they say, oh, you don't mistake my knowledge with you with the knowledge that you find online. So, how do how do we go about that?

Because I think there's a huge, huge, huge opportunity in the fact that people are looking for these answers through the chat GPTs and what have you. And then there's also a danger related to it, but also this mismatch between what I call speed one world, which is the outside world, and the speed two world, which is a lot more slower, which is the healthcare world. Yeah. I mean, I'll start by saying I think I'm entirely in support of people taking a more active control of their health and wellness.

And I personally also use AI in this way too. You know, it's coming from the right place. You know, people want to play an active part in that. And often the difficulty in being able to get the care that they either need or they want will sometimes drive them into it.

I think the other thing as well, Christophe, is that remember that people, even if they say I don't use AI, if they use Google, yeah, you're using AI. I mean, it's swung towards like AI first. So, you know, I mean, I think most hopefully most people will recognize it, but maybe some people don't. So, yeah, it's fine.

And, you know, again, that principle that I was saying about like, I want the jetbacks on the moon, I want all that stuff, but let's see what we can do to minimize the likelihood of things being harmed. And the same with patients and people and their families is what can we do to help them do so safely? Part one is working with the manufacturers and the people building it and trying to get them to think just with a little bit more care about how they talk about things, about the capabilities of what they do.

And you'll see it with chatbots from ChatGPT and Anthropic. You know, they have some guardrails. They do, they do put information in. They may not be perfect, but they are pretty good.

The challenge is, I think people don't have that baseline understanding of what these things can and can't do, and therefore where the risk might be. And so they might not see when something risky is happening. I think, speaking from my side, I think just as we did with the rise of the internet, there's a piece of education and acceptance and helping people and working with them. So when they want to do this, they feel comfortable in talking about it, so they don't do it in a sort of sneaky way.

They're a you know, they don't want to upset the doctor, it's fine to talk about it. Finding a way to weave it in and assist them, but that requires the doctors as well to understand it too. So education's a huge, huge part of it, I think. I do a lot of work trying to get AI taught in medical school, for example, because it's not.

Yeah. No, I think what you're saying is right, but what the I mean, if you look at the two speeds, I come back to that. Speed one goes way faster than the speed two. Physicians are not the often not the early adopters, then there's the safety rules, then everything.

Also, the I've studied this, this is my education, you don't know anything, kind of paternalistic protectors. The terrible mug, the you know, don't yeah, the terrible mug. But also the protective paternalistic thing is still a bit, you know, it still lives in the healthcare industry. And so one of my keynotes is called how AI flips healthcare upside down.

And so if you take these two speeds and if you look, you put them three, four years, maybe five years into the future, then you'll have patients using AI, which have become probably more intelligent, I wouldn't say smarter, but more intelligent than a physician. Would that flip, in your opinion, the dynamics? I always talk about the age-old dynamics in healthcare, is like information and solutions that were primarily reserved for the healthcare people, the shamans, the priests, whatever.

But now with AI, we have a democratization of knowledge, information, tools, even actions that you can take. So it kind of completely flips the dynamic that has been around for two, three, ten thousand years, if you will. Yeah. So how do we how would we go about that?

I mean, if in a couple of years that dynamic is felt in some way? Yeah. How do you prepare the healthcare industry for that? So the first the first thing to say is that I think it's overdue for that change.

You know, I'm actually supportive of that. And it's going to be choppy as we work it out. Exactly what is valuable about the shaman, the healer, the doctor, the carer. I mean, I also believe that there is something very important, and whether it's unique or not, but I think there's something very, very critical about that role of persona that's played by a person when you have two people coming together to talk about their care.

How do we deal with it? The first is that we don't deny that it's gonna happen, right? Okay, we don't stick our head in the sand. I mean, if you do that, that's many do still, though, I think.

Oh my gosh, yeah, they do. And you know what? Your choice. But I'll tell you something, you what you're doing is you're taking you're playing no part in what comes next.

And what's coming next is coming, right? So that's the first thing. The second thing is that if you do start engaging with it, there'll be some aspects of fear and there will be some things that will change. I'm talking about the healthcare establishment side of things.

But if you engage, you have a part to play in shaping well, what is valuable about what we do. And I talk a lot about we've given up speaking as a doctor here. We've moved so far away from something really important because we've been diagnosing, studying, transactionally delivering high-quality care at scale and everything like that. And we've been seduced by the idea that is what we're here to do.

Now it's an important part of what we have to do. But if that's all that we do, and then a machine comes along that can do that fantastically well, you're right to be scared. But that's not what a doctor does solely. We also care for patients, we also bear witness and help people measure and understand things.

We communicate, we become part of their understanding of their health and what they should be doing. This is something that, in the absence of technologies and knowledge, we did a lot. So you mentioned shamans and healers. I talk about this a lot.

There's something about that role, and these are groups, even in modern society, that continue to cleave to what might be perceived as an older way of doing things and it's maybe dismissive. But that is incredibly, incredibly powerful. And I think there has an opportunity now for us to hopefully rediscover and lean back into that alongside the important technical parts, assisted or managed by AI. We all in the healthcare system have to actively engage in that because if we don't, it's going to happen anyway, and it may come out in a form that is maybe less good all round.

That's my general feeling on it. I get it, and that's a very safe answer, I think. Alright, that's fine. Happy to be challenged.

Yeah, yeah. I think the one thing that I'm wondering is that so I once did this exercise, and I think I told you already before, maybe in a previous episode or maybe offline, but there's this one thing where I did an exercise because it's hard to see what how AI is changing everything around us. And so the exercise that I did was what in hindsight, and in what way will we look at AI in hindsight and how it changed healthcare? So imagine you're 10 years ahead.

What would be the changes that you didn't see happening because you were in the midst of it? And I think I tried it to formulate them in I did it with AI, it was a chat, it was a chat with AI back and forth. And one of the things that struck me really hard, but really totally made sense, is that I can't remember the exact quote, but it was something like, How did we ever trust our health to the ego of a person instead of the intelligence of a tool? Hmm.

Yeah, no, I see what you mean. I mean, maybe well, we've had this a few times, haven't we? It's I think so. Again, I'll speak from the Because you were talking about the role of the identity and the person and so forth.

So that's why I'm picking up on that. Because the question would then be what is the role of a of a physician or any H E P then in those kind of conversations? Yeah, I suppose so. If I go back, you know, again, this is an oversimplification, but if you think about the trajectory of a human life and you start out as a very vulnerable baby, infant, child, and then growing into hopefully an adult that is, you know, self-determining and operating safe, you know, safely.

Yeah, well, operating in the world and interacting with other people. But for you know, a very substantial portion of our life, we're extremely vulnerable. And we require care, you know, not just healthcare, just like feeding and clothing and so on. So that's very much part of the human experience, the sort of experience of life.

And as time goes on, you'll acquire enough to sort of operate independently and make some mistakes, but not have you know terrible mistakes and so on. And but the world is big and knowledge, and no one person can capture absolutely everything that they need to know to operate in the world, hence the arrival, you know, like experts and so on. Of course, AI challenges that because that gives you the promise of like, well, that thing that the expert knows, which you couldn't have access to, now you can in the form that you want, and so on.

And maybe that will change everything utterly profoundly, and in which case, what I'm saying here, who knows, right? the closest I can get to it though, is well, if you look at where AI is now, the experience of people working in healthcare with AI right now, I have to tell everyone, is well behind the curve, right? I mean, we know this. You talked about speed one and speed two, didn't you?

You know, technology moves fast, health moves at another rate, because there's a period of time where correctly you're trying to work out can it be used, how it's going to be used, how does it fit with everything else, you know? But that pace, speed one, is now speeding up, and speeding up, speeding up, and speeding up. And I sometimes talk to people about saying technology's achieved escape velocity, as in healthcare will never capture it. Like it's escaped, right?

But that's okay, maybe, because you know, as long as you use it usefully. But I think I one correction, I think the right word is healthcare system will never do it. Yeah. Because healthcare will be provided in different forms, maybe, but the system, the approved system, the reimbursed system, the educated system, that will never do it.

But healthcare will be maybe in other forms might exist. Yeah, you're right. Actually, and you're and you're right to pick me up on that. Remember, remember that this is a big business and industry, regardless of how it's configured, that confers great benefits on no participating in it, including power, financial security, recognition, and so on.

I'm not saying that because, and that's an evil thing and people shouldn't do that, but it just it just wherever you are, we do things for certain reasons. it is world straddling, very important, and has a lot of inertia, you know. Like, why we, you know, we look at the innovators' dilemma, why would a well-functioning thing, well-functioning for the people in it, maybe, change? So we've got that kind of counterpressure as well.

And maybe how I'm speaking is I'm part of that counterpressure, which is like, how can we take this existential threat to things that keep me roof over my head, and find a way to keep me in the loop and therefore keep the roof over my head, but make things better. But you're right, there is within that the possibility of a better way of doing things that is radically different. And actually, funnily enough, if I don't know if I've got it here, but there's a here it is. There's a there's a really good book that kind of approaches it that way from someone called Charlotte Blease, called Dr.

Bart, that directly addresses what you've just said about that, which is extremely assertive about the need for things to be radically different, and a very, very good argument for it as well. I won't go into the whole book because that's a great book. But the question is whether this happens in a moderated way, as in people move, or is there going to be a sudden remember when we first met, we talked about revolution? Yeah, yeah, yeah.

What would revolution in healthcare feel like? And I think at that point when we first met, and it was in just after COVID, actually, wasn't it? We didn't have the tools that we have now. So it was an interesting thing to discuss.

I think now, if we were back in Toulouse going for a stake and having that conversation again, I think we'd probably have much more concrete thoughts as to the reality of that happening and what shape it might take. And I think, of course, AI and people taking the care into their own hands will be part of it. One thing I will say though is if you watch The Pit or anything like that, it was a very, very strong reminder that as fantastic as AI, people still need to have be pulled out of cars and have operations.

Now, robotics may address some of that too. Sure. Yeah. But I was I was very, very quickly reminded that sometimes there are things that happen in healthcare that possibly aren't in the immediate future going to be completely replaced.

And within that, I think a human bearing witness to another person's suffering and saying it's okay and guiding them through it in a physically present way. The question is, how long is that going to be important? Is it always going to be important? Or is it just going to be good enough that a person that you really like and can care for you in a human way, with all the AI giving you the medical stuff, is that going to be enough?

I think that's the what that's the wild bit that we have to watch. And all I would say to people in healthcare is don't turn away from it because it's scary. No. I think, first of all, coming back to the what how would revolution look like in health?

I think I think I still remember the answer that I gave you was that consumers or people will take a greater responsibility and knowledge base and will be will be driving everything. I mean, this is everything behind health reason. That's what I've been doing for the past 12 years. This is my conviction of a revolution.

And we didn't have such a view on how AI would impact that. I think the only thing that we see now is that it will accelerate even more than I could have expected. Yeah. Now, coming back to your point, what is the role of the physician and how will that change?

I think that is one of the things that I, even for my own business, I'm thinking about a lot. And the one thing, I mean, if we look at the facts, the expected facts, and how the world would look like is that AI will be smarter than us. That's for sure. It is, it is already.

If we look at the there was, I think Eric Topol was posting yesterday something about or last week, something about the new Google model being more, you know, liked by patients after the third visit. So we see, and these were even old models, so they're even redundant, but so the empathy and relationship people will trust. We see already studies that people trust AI more than physicians. So and the same is for my business in speaking.

I mean, right now I've been I was always booked because people like my opinions, the way that I reflected upon stuff, but also that is driven by ego in a sense. They trust my opinions. Will they trust my opinions over the AI? Probably not.

and of course, I need to use AI, physicians need to use AI to level that up. what is the role of the human being really? If empathy, intelligence, even reasoning will be if we are less smart, less you know, emotionally smart. What will be the role of the physician or the human beings then?

I mean, I have my thoughts on it, but before I drop them, yeah. yeah, I mean, like, so we can discuss this and put our thoughts in here as well. I think it's also important to say that we could both be wildly wrong and it'll be something else. And you probably you know, it I think it's important to approach it that way, but it's still instructive to sort of think about how you'd ask this right now.

The maybe another way to sort of think about this is I remember working on there's a lot of work done on technology to help people care for themselves, yeah? And increase engagement and so on. But there's a really simple thing to help you look at it a different way, which is the bulk of this healthcare spend is on the sickest people, people that can't care for themselves, either people that are acutely sick, or people that have the inability to care for themselves, or if they're maybe young and they have, you know, they're children or infants or anything like that.

So I can't put a number to it, but I think you could possibly broadly agree that the people that need the most healthcare are the ones that cannot deliver it to themselves. So if I was to say to you, a person in intensive care should use Chat GPT to adjust their drug doses, right? People would say, well, of course, that's absolutely absurd because they're unconscious, right? Okay, yeah, but there may be sort of ways around to assist with that.

Or, you know, a person's crashed their motorbike and they're lying on the side of the road, you know, you know, they would start typing into Chat GPT, you know, I'm injured, what do I do? You know, like those are absurd sort of end points to that. So you're always going to need something else to care for the sickest people, and parts of that could be done outside the physical world, bits of it, the knowledge and everything like that. The question then is first of all, just practically speaking, how is that physical side of things dealt with?

And then there's the deeper one, which is and do we care that is a person, a human, or not? And we don't know. There is a sense, like you said before, that we're we're changing because the use of machine is changing what we're happy with, you know? Yeah, also I mean, we're talking five to ten percent, maybe of everything of all the you know, the actions taken in healthcare.

We're you're talking now about four to five to ten percent, maybe. Maybe twenty, but it won't be more than twenty. Yeah, yeah, yeah. The question is how my question was more oriented towards the 80% or the 85 to 90 percent, which is the daily conversation that we're having with physicians, the daily checkups, diagnosis, follow-up prescriptions, bloody da.

I mean then that Christoph is completely up for grabs, and if I'm honest, will be fully handed over. Fully handed over, okay. Yeah, and the trans, like so the health care, health care, you know, you can say care is human, health is machine. That's a really reductive way of doing it.

And then the bits that the technology can't reach are then handled somewhere in the middle. And maybe all of it will be done and what will I have though. But so can I see a world in which let's say five years from now, it could be sooner, could be later, where the idea that you would go into your doctor to measure your blood pressure to get them to approve a blood pressure medicine and have it prescribed to you could feel very antiquated. The answer to that is, of course, yes, because that's already happened in Utah.

And so what happens in a small way in one place is of course going to happen. We didn't need AI to do that. We could have done that with a simple rules algorithm beforehand. There is some benefits of me seeing a person with blood pressure problems to do that, because other health issues can arise and I can observe things and so on.

But if you're talking on a global basis, I think the utility of being able to, yeah, if we could control the world's blood pressure, we would manage the leading cause of death, which is cardiovascular. A machine could do that, maybe. So the way that I would add that the way that I try to answer that question, you know, what is the role of a human being when AI is more intelligent and emotionally intelligent than humans? Is something then I go into the realm of spirituality, which I mean I follow, I'm not too naive or too big in it, but I think there's one thing that we have between humans, and I'm always wondering how we can translate that into the future of the world.

The one thing that we have between humans is energy, like the energy that you feel between people. We're sitting here virtually, so even we have some sort of energy, but if we would do this in a bar with a steak next to us, like we did a couple of years ago, I mean, the energy would be completely different. And that cannot be replaced by a zoom call, riverside call, that cannot be replaced by an AI, or at least not in the same way, because we have the serendipity, we have the microexpressions, we have all the flaws, we have the, you know, the energy comes from all that, which will be different with an AI.

It's it will be artificially clean, artificially perfect, artificially whatever. It will be artificial, not real. And so I'm always thinking about what does that mean for healthcare? What does that mean even for my business?

I'm a speaker. Why would people want to have that energy with me? What why would people want to be listening to me instead of just a robot who probably speaks better in English than I do? Almost like I definitely would speak better in English than I do, have smarter ways of putting it in a crisp way, and so forth.

So, why would people want to listen to me? Why would people want to go to physician? I think the answer lies somewhere in that energy between humans that we will probably be craving for. I always make that comparison with social media.

Social media was built to connect people, but what it did, it created more loneliness because people feel less connected than ever. And I think we will see something similar with AI as well. Is that it is smart, it is emotionally savvy, it has manipulation techniques, psychological, blah. But will we crave something else then?

Well, what is the thing that will come out of it that you know AI will take away from us? And it has to do with something with energy. And I'm wondering where what the role of an HP or a clinician or whatever would then be. And will that change the way that you know the task force and the and the and the healthcare world will change?

Will the type of people that choose a healthcare profession will be different and so forth? Yeah. So before I answer the med the health part, we know that you're a musician, Christoph. And you sing and perform.

And you'll go and see people singing and performing, yeah? Yeah, yeah, you could hear it better on a recording, no doubt. You could watch a video of the show and see it much more clearly. Exactly.

Yeah, people still do. And why is that? There's two sides, I think. One is that being present with the performers, you're part of the performance.

You may be a the symmetry. Being present is a nice way of putting it, by the way. Yeah, you're part of the performance, and then you know, you can say, well, there's the imperfections and how it does and everything, but that's part one. But there's another part to it, which is you're there with other people, you're sharing the experience too.

So those are two things. There's music, or cinema is another example. I can watch it at home, but I like going to the cinema because I like sitting in a dark space with a big screen, but the other people are there and it's the event. I don't think I think it's it's prideful for us as humans to say that we know absolutely everything.

We know a lot about stuff. I think that we can know everything about everything is wrong. So whilst I'm an atheist, I increasingly call myself an agnostic because there's, you know, my intelligence and our intelligence is only so far. I think there's more to it.

And what you're saying, I know it feels nebulous, but issues of energy and physical touch and so on, there's enough out there to suggest that there's maybe more going on. So there's all of those things. But then I again come back to, you know, I've told the story before. I when I was a student, I broke a finger, I was in hospital, technically everything was done well.

But the one thing I remember from that is when I was recovering and in a bit of a sort of post-operative stupor, I got tangled in my bedding, and a nurse came up and untangled me and said, Look at the mess, you're in sort of thing. But she turned my pillow over from hot to cold, and I still can almost feel her presence, and I felt better. It's that, and actually, that's the bit that we've walked away from so much in healthcare, and why I think we have technically great care, many people, but don't feel cared for.

And I think can a robot like with a robot flipping my pillow feel the same as a nurse coming up and having a bit of a banter and flipping my pillow? And could it be so I mean, I don't know. I feel that it couldn't. And maybe the reason for that is not because we haven't got the technology yet, but because there is something more going on that you've mentioned here.

And I'm not gonna profess to say that is impossible. So, again, what do you do in the situation? Let's stay open and willing to change and humble in the face of our own experience. Yeah.