Paul Testa, MD, Chief Health Informatics Officer at NYU Langone Health

- Every health system executive right now is being - asked to do something that has no useful - precedent, - make durable technology decisions in an environment that - looks meaningfully different every six months. - The pace isn't slowing, - the vendor landscape isn't getting simpler, and the - cost of missteps - keeps rising. - Becker's eleventh annual IT and Revenue Cycle Conference, - The Future of AI and Digital Health, exists - for exactly this challenge. - Here, you'll find more than 2,500 executive level - attendees, - 600 plus speakers, and four days of sessions - built around the questions leaders are actually wrestling - with.

- How to scale AI responsibly, - how to protect increasingly complex systems, - how to bring more performance out of the - revenue cycle, and how to make tech investments - that hold up over time. - Join us September fourteenth through the seventeenth at - the Hilton Chicago. - You can register on our events page at - beckershospitalreview.com - by clicking on the events tab in the - upper right.

- This is Laura Dierda with the Becker's Healthcare - podcast. I'm thrilled today to be joined by - Paul Testa, chief health informatics officer at NYU - Langone Health. Paul, it's a pleasure to have - you on the podcast today. - Thanks for having me.

Always a pleasure. - Absolutely. Now I'm excited for our conversation. I - know we'll talk through some of the cool - things that you're doing as well as perspective - on the future.

But before we dive in, - can you tell me a little bit more - about yourself as well as NYU Langone? - Happy to. Thank you. - So I'm a born and bred New Yorker, - which, you know, really identifies a lot of - who I am, I feel like.

I've been - at NYU Langone now for twenty two years, - give or take. I did my residency in - emergency medicine there. - And, - it's it's been home for me for a - very long time, so it's been incredible to - watch it grow and, - and excel. - I, - right now, get to the privilege of leading - health informatics for the system, - which has been really, I think, a strategic - differentiator as we grow, as we expand, as - we bind - ourselves together as a large academic health system.

- Informatics is the glue that holds us in - many ways, and it's it's been really incredible - to be able to watch - as we - continue to set quality - standards for ourselves - to leverage all the technologies. And then, you - know, let it let it mark the one - minute moment where I mentioned AI, - how we've sort of heavily leaned into AI - and what that's meant for us as a - health system. - So it's been a great venture, and, - I'm loving what I do. - That's fantastic to hear.

And, you know, to - your point, just having that data information informatics - and and translating that into something that's beneficial - across the board, - is so important in health care right now - and something that is a huge unlock for - organizations once they once they can do it - well. - Now what are some of the powerful forces - that you see shaping digital health and IT - at NYU Langone, and how do you anticipate - it'll evolve over the next year or so? - An incredibly powerful force, I think, - has to be the economics of what we - do.

- We need to be really conscious stewards - of the resources - that we have access to and deploy. - It's - it's - not as challenging to deploy 10 of something - or - to have a small innovation project that has - great - results in - one location or five locations. - But to steward our resources and to get - the the bang for the buck - as we - are in sort of every facet of the - way we deliver care, you have to do - it at scale. So real innovation in my - book is anything that we can do at - scale successfully.

- And - the - the cost of - what it will take to - innovate, - to do AI at scale, to continue to - drive the quality and safety, you know, and - most importantly or or the binding is the - the digital patient experience partnered with the digital - clinician experience. - We have to do that understanding what our - success metrics are and what our return on - investment is. And there are multitude of ways - of measuring both those things. - So we're we're setting out a standard internally - at NYU Langone - to - to measure like for like.

So when we - deploy a project, we look at what our - measures of success will be, and then we - try to use the same measures for the - next project where we can so we can - compare apples and apples - and look at the way we spend our - resources. - I think all of that is in service - of our patients and our clinicians - and ensuring that it's an absolute, just incredible, - joyful digital experience - in some of the trying times of our - patients and our clinicians' lives. - That makes a lot of sense.

And, you - know, I I think it's so critical - to be able to have that great patient - experience because right now, it seems like there - is more and more of a need and - demand for things to be - seamless across the board. But especially in the - health care space, there could be a lot - of complexity. So how are you seeing, those - solutions being, - driven at NYU lingo and health? - What are the ways that you're seeing patient, - preferences change, and how are you meeting those, - expectations on the technology as well as the - clinical care side?

- Oh, I love that question. Thank you. The - simultaneously, - patients expect, rightfully so, - the most cutting edge, - highest quality care. That's their given right and - what they show up at our front door - expecting, and we have to give that to - them each and every time.

- But they also expect, and we want to - give, - the most empathic - connections and care. - And sometimes - when technology is deployed, - maybe in less than a thoughtful fashion or - without the patient and the clinician front and - center, - that empathy can suffer. - So we're finding so much, for example, with, - with leveraging AI to create patient friendly versions. - I love us calling it patient friendly.

It's - all patient friendly. The fact of the matter - is we've been writing notes for decades. - We never and the patients weren't our primary - consumer, so we're shifting, and we're gonna figure - out what it means. But we created a - rubric - at NYU, went on clear rubric - of defining patient friendliness in five different domains.

- So everything we put in front of patients - has to pass that rubric as well as - an AI safety checklist and make sure it's - ethically and safely deployed. We also have to - ensure that it's - ingested the way our patients want it. - Be it - in a you - know, they're finding the patient's report and tells - us quite quickly - that the AI generated summaries - can be, - or and AI generated - versions, - the patient friendly versions, everything from our - e e, our echo results - to our discharge summaries - to every day, giving discharge the the summaries - of a patient's stay in the ED in - the, inpatient setting.

- Patients - are finding the - the AI generated versions to be more empathic - scoring. And I'm not gonna say that's a - bad thing from our docs, but we we - optimize for something different with the doctor's outputs - and the nurses' outputs, - and this gives an opportunity to tune in - a different direction. - So I think - meeting both their safety, quality expectations, - as well as the experience, that's our challenge - here. - That makes a ton of sense, and I - love the way you laid that out so - perfectly because it is, such an important thing - to understand what the patients are dealing with - and how they, - are are wanting to have the experience of - health care and then exactly what AI can - do and and how then the caregivers can - lean into that, relationship that they have and - what they can, do to most helpfully, - bring the patients into, - recovery.

- Now from your perspective, could you tell us - about something you're doing at NYU Langone Health - that, - has been a big swing you took in - the last year that paid off? What did - you do, and what were the results? - You know, I think the biggest swing we're - taking right now, and I I don't have - I don't have all the outcomes yet, but - I know where we're going, - is, - you know, there's so much, - inches of headline and conversations around ambient documentation, - how important the ambient documentation is for the - experience of patients and the doctors.

- But what is - trailing, and I think it'll be more revolutionary - to the experience aspect, is nursing ambient documentation. - So we're deeply involved both with our EHR - vendors as well as third parties and our - own efforts internally with our own AI expertise - of what it's gonna mean for nurses to - be able to practice out loud, to speak - their practice, which they've been doing for centuries. - But capturing that rather than having to run - to flow sheet rows to document.

- I think, you know, the - the the space of an exam room - with a physician is a very different experience - than an inpatient setting in a room with - a nurse. - I think it's we we have seen - incredible gains - with ambient documentation - for physicians. - I can't wait to see what we're gonna - have with nurses using this tool at scale. - So we're we are early in that journey, - but I think it's gonna dramatically change the - practice - as well as how nurses are able to - connect - with their patients and manage their day.

- So I I have incredibly high hopes. I - think the - the way putting it is, you know, the - the win for Ambien and the physicians, and - it was an easy one to see. - For nurses, it's gonna be a little bit - more it's gonna be a lot more effort, - but I think the, the incremental benefit is - gonna be incredible. - That makes a ton of sense.

And, you - know, it is cool to have that opportunity - to then support nurses, - who are such a critical - part of the the patient care journey, to - then have that ambient support there too. So, - you know, when you look at that change - management aspect of it, as you mentioned, doing - nursing out loud and and being able to, - fit within the workflows, - how do you approach that? Obviously, like you - said, with physicians is one thing, but, - what is what do you do to make - sure that, you you know, you're working with - the teams and it's as seamless as it - can be?

- Oh, it's a tricky one. - And and yet in some ways, it's incredibly - simple, and that is to meet the nurses - and patients where they live and where they - are. - And the very first thing we learned - is - nurses, - the units that we had been using, our - proof of concept for ambient - documentation - were quieter, - frankly. They were used to engaging in their - practice - in a less out loud way.

- I'm not gonna take any hits on that - as an ER doc. I'm I'm kinda I - can get loud myself, but I think we're - more practiced, and there's an expectation - of - speaking as we go. - So, - with our CNO, - and our informatics - nurse leadership, - We engaged on a project essentially, practice out - loud. - And it it - while every one of our nurses has a - clinical mobile companion, an iPad with a suite - of apps, - right now, - it was a step we could pave the - way without any technology, and that was simply - peer to peer engaging - in what it meant to practice out loud - and having nurses - assess themselves - and what speaking their practice and as they - engage in their assessment, - means for them and getting used to it - because some took to it very quickly, and - some are still working on it.

So it - it was we we engaged in this practice - out loud program - irrespective of technology, just just just to move - them towards, - reengaging their own voice, - as nurses at bedside. - And it's been incredibly, - enlightening to watch. - I don't wanna say - a a renewed voice because it I don't - think it went anywhere, - but it's stretching different muscles, - flexing different muscles for many of our nurse - colleagues - that are really leaning into it. And it's - it's incredibly transformative - to what we're hearing bedside.

- That makes a ton of sense, and thank - you so much for sharing that with us. - I think it's so helpful, - for all health systems as they're going through - some of these changes to understand what's working - well and what can make a difference. - Now that, you know, you you've been working - on these projects and getting technology in the - door for both the patients as well as - clinicians, what's the most important problem that you're - trying to solve next with AI or technology?

- The I I I won't say easy wins, - but the early wins, - with the application of artificial intelligence and generative - AI in health care has has had a - lot to do with what we've spoken about, - summarization, - extraction. - But, - you know, as an ED doc, the magic - is in the flow. - So we're beginning to inject models - not just at the extraction point of, why - is this patient in front of me? - But - or or what transpired in the last twenty - four hours to six months with this patient, - be it ambulatory or inpatient.

- We're looking for the the wins to be - had in the flow of an ED, the - flow of a hospital, the flow of a - patient in the arc of their care. And - that is sort of I think, you know, - it's it's it's often referred to as longitudinal - decision support. I don't think that really captures - it, - But how do we - have models which are ever present or can - be ever present? And they can be - always, - vigilant - as a copilot, as we often hear the - term, for nurses and doctors - in their day to days and in the - week to weeks.

So we've deployed models in - the emergency department that are looking for high - risk diagnoses that can get - missed or - anchored away from with an earlier diagnosis. - And we're reducing that model so it's ever - present watching the documentation, - watching - the inputs from lab results and for imaging, - and then - providing a nudge to say, hey. Maybe it's - time you know, we can see in your - documentation you're thinking the diagnosis is x. - The risk or the likelihood of it being - y has now hit a threshold.

And I - wanna just drop you a nudge to say, - reassess. - Now that is that is bread and butter - ER practice. - But I think any any, you know, any - comfortable, - ER doc will tell you that we can - anchor bias. So - we welcome those nudges - when appropriate, when well tuned.

And when when, - frankly, they aren't imperfect. We're not looking for - a diagnosis machine. But the opportunity to offer - a nudge away from anchoring bias is incredibly - powerful. And frankly, it's a it's a - blanket of security that the patients and the - nurse and the doctors are finding together in - the ERs.

- That's incredible to hear about it. And and, - really, it seems like it'd be helpful, - across the board for folks in in the - clinicians working in the ER as well as, - you know, the the patients too. So, thank - you for sharing that example, and and we'll - be excited to hear more about how it - goes as you continue to roll things out. - And, you know, in in thinking about the - future, what are you most curious about or - or what rises to the top of some - of the things that you're keeping your eye - close on as time goes on?

- Most curious about that. That's a that's a - great one. I - as the evidence mounts - that - having a clinician in the loop may degrade - performance, - Inevitable. It will it is inevitable.

We will - find circumstances where the addition of a - clinician - degrades an AI model's performance. - And the ethical quandary of when we - maybe need to remove the person in the - loop to ensure higher quality. - We're getting really close to that moment. - Am I suggesting we should set these models - loose right now?

Absolutely not. We have a - lot to learn and our patients expect for. - But I'm very curious as to how - the, - frankly, - disruptive - nature to our - definition of professionalism and our definition of who - we are as clinicians - is gonna be shaken - When there are moments we need to step - back and say, - the central thing for me to do is - to let this - process run and mean not to interfere, but - still have the courage to say, I need - to validate. I need to be there.

- So - more complex way of things just saying, human - the loop, we're gonna learn isn't always always - the protection we think it is. - And as we're learning with these models, - I'm very curious as to what it means - to be a clinician and how we're gonna - define our role working with these models and - with these tools as is almost every profession. - I'm no word about being replaced. - And if it means to be the provision - of safer care to replace me in that, - then so be it.

Replace me. - But I I think what I'm most curious - about in the next one to three years - is how we move into that space of - acknowledging, at times, the performance - of these models - outperform me, - outperform a team, - and therefore, we have to understand what its - role here would be moving forward. - Well, that's such an interesting question. And, you - know, it sounds like you see that horizon - not too far off in the next one - to three years as as the technology continues - to evolve.

And, you know, when you're looking - at it, - what types of discussions do you have with - your colleagues around this, or how do you - really see, - the humans in the loop being able to - make that estimation and and figure out what's - gonna be most impactful for patients without putting - them too much at risk? - Well, here's the great thing. - As a profession as professions, - doctors and nurses have been assessing - what tools we bring to bear for the - betterment of our patients for century.

- So I think we're we have the rubric, - if not - the line by line assessment tools. - We have a model of questioning, - proving, - obtaining trust, - and deploying. - And that's what I'm hearing from my colleagues - is we're still in the trust building phase, - and I think that's appropriate. - And we've learned a lot from basic CDS, - from EHR implementations, - from radiology - at this for thirty plus years, from - from the digitization of pathology, - in the last year, many of the most - recent years.

- So we're we're we're we're reasonably good skeptics - with an ability to prove, and we have - a scientific process to do that. - I think we're definitely still in the in - the trust building phase for both our patients, - and our on our peer clinicians. - And that's what they want is an explanation - of how these things - are doing what they do - and when do they really feel when they - earn the level of trust that we can - hand over some of the control. - Again, we're not there yet, but I'm not - I don't think it's five to ten to - fifteen years away.

I think it's it's much - closer than that. - So the reckoning is coming is coming, and - that reckoning will only be, - resolved or acknowledged or moved past once we - build the trust together. - That's fascinating to hear. Paul, thank you so - much for joining us on the podcast today.

- This has been such a fascinating discussion. I - think you've shared some really unique examples of - ways technology and AI is is, - helping the organization and really making and and - transforming patient care and then looking into the - future, what the possibilities hold. So - thank you so much for your time, and - I look forward to connecting with you as - well in person at our health IT and - revenue cycle conference coming up in September. I - know you'll be speaking on a panel, and - it'll be great to kinda dig deeper into - some of these themes, and, - I'll enjoy seeing you there.

- Thanks for the privilege, Mark.