Emotionally Intelligent Innovation: Transforming Customer Experiences with Craig Tucker

Speaker A: You're listening to the Science of cx, a podcast that hopes to inspire business owners and leaders to learn new techniques and turn prospects into customers. And turn customers into raving fans. My name is Steve Pappas. I'm known for my relentless pursuit of all things customer across my career and also in my six startups. Um, I've had to learn how to make decisions in business that customers really respond to. Spend some time together and help your business soar, grow and accelerate.

Speaker B: Well, welcome everybody, to another episode of the Science of cx. I'm Steve Pappas, your host, and once again, we've got a incredible show for you. Not just great, it's going to be incredible. And let me tell you why. Because it occurs to me, being in CX for as long as I have been and in customer service and in walking in and out of hundreds of contact centers every single year, that the current technology we've all been using called sentiment analysis, is just not cutting it anymore. After all, what do you do with positive, negative, neutral? If somebody says to you, well, it's going positive, it's going negative, it's going neutral, do you really know what area it is? What problem has presented itself, how you should counter the objection, how you should deal with the problem? It's always been problematic. And I'm not saying there are a lot of great sentiment analysis companies out there, but it's never been able to take us far enough into the mind of the customer, or the prospect, for that matter, because we could be talking about sales situations, too. But I personally would like to know what emotions the customer is exhibiting. I would love to know, when does the conversation change? When did they get that frustration point? When did they get to the point where in their minds they might be saying, yeah, okay, whatever, and thinking, I'm just going to go find another vendor, I'm going somewhere else, I'm ready to bolt. This stuff is so important. And I've looked out there, I've looked at all of the players in the marketplace, and we've invited Craig Tucker, the CEO from a company called Vern AI, and what you're going to hear today is really going to blow your mind. The reason is because if you think about contact centers in general, or phone conversations, text conversations, bot conversations, the one gap they all have is the fact that you don't know how the other person is interpreting something. How are they perceiving something? How are they feeling? What is their emotional state? And isn't that something we do know when we're sitting across from somebody, all those signals, all that information, from body language to various signals, to the smile on their face, to how their eyes smile, to all of the components. We give those signals to everybody day in and day out. But if you have a phone conversation, you don't get the benefit of that other half of how we communicate and how we communicate what we're thinking, what we're feeling, what our emotions are. So whether it's bots, whether it's text chat conversations, whether it's your Frontline taking calls, they all have a gap. We're going to talk about the solution for that today. And I am just loving the fact that we're able to get Craig Tucker, the CEO and founder of Vern AI on today. So Craig, I want to thank you for joining us today.

Speaker C: Thank you, Steve. It's wonderful to be here.

Speaker B: Well, let me give folks a little bit about your bio. So Craig is the creator of Vern AI and you are going to hear a lot about this. The Virtual EMotion Resource Network, Vern for short. Craig Tucker focuses on emotion recognition system. He has a Ah BA and an MA at Ah, Michigan State University, including three year PhD study and he's a lifelong Michigan resident, enjoys living in the Great Lakes State, but has truly hit upon something that I think is going to be one of the next big areas. If you thought ChatGPT changed the world, imagine if now we can understand our customers, our prospects, those we're trying to sell, those we're trying to service. If we can understand them at an um, emotional level, how much better can we match, uh, our products and services to them? How can we match how we communicate so much better? So Craig, let's start out a little bit, if you will. If you could just give us a little background on what was your vision, what was the genesis of this? How did you think that this is a problem that needs to be solved and you're going to solve it.

Speaker C: Great question, Steve, and of course, thanks for asking. Sentiment analysis has been in use for almost a decade now or more in various types of conversational design. So the complaint has always been is, yes, great, I know it's positive or negative, really know what to do, a neutral or mixed, but still don't know if somebody is happy or sad, if somebody's trying to use humor or if somebody's fearful. And so there was a huge gap in the literature and the science of emotions itself because one of the reasons is psychology has different models. A standard model of 5 or 7 depending who you ask. It can be 9 or Plutix wheel of 13 emotions which aren't a continuum and diametrically opposed, which unfortunately is really not how the brain works. But there's even models that say they're 27 and 40 different emotions. I don't need to be a data scientist to kind of know that all of those different models are going to conflict at some level. And so that when we have automation to detect these emotions, we need to have a model that is agreed by most people. Unfortunately, psychology can't agree. So we're left with a uh, giant chasm, this gap, if you will. Good thing is there's other disciplines, there are other sciences, including neuroscience, which does allow us to look inside the brain using FMRI and other techniques, actually see the stimulus and response in near real time. It's not perfect, but it's getting better. That's allowed us to kind of do uh, some deep dive into the actual biology of the brain instead of looking in the black box, which psychology does. And we follow a lot of Edmund Rolls's seminal work in neuroscience where detective, positive or negative is more like euphoria and dysphoria and there's this weird little thing called fear and some things called a prediction error, which all kind of roll into conceptualization and neuroscience of what emotions are.

Speaker B: Uh, I'll tell you, I'm fascinated by this. At first when I was turned on to what you guys do, it was over my head. I was thinking this just seems like it would create a lot of chaos for customer service agents or for conversational design for bots, or in text based customer service, et cetera. And then I realized that the way you distill things, you distill them in such a way that it doesn't add any chaos to the customer service person. What it does is it gives them so much more insight that it allows them to take care of the needs of the customer so much faster. And it allows them to kind of match up their talk track and everything else that they would normally do and do well, but it gives them that extra. So I can imagine that even average handle time goes down, first call resolution gets better. Customer success scores, whether they be NPS or CSAT or any others will get better. But maybe you could take us through a little bit on how Vern works, how the system is actually works and what is it looking at and how does the software give all this insight in real time? I think it is too, right?

Speaker C: Yes, that's correct. And uh, that's actually a really good question. Long story short is we Created something that's a little bit akin to back to the Future back in the 80s when all of us old guys and gals out there really and relate to that one. But no, it really is. Back in the 1980s, we started to do a lot of research and development, we as in the community, into artificial intelligence. And we came up with some systems and some processes that seem to work for the time, but we're lacking because of the computing power of the time, kind of put back on the shelf and ignored and forgotten about. And what I'm talking about is something, uh, akin to a fixed expert system. And anybody is old to computing knows how that works. So Vern is not that it is a much different and hybrid approach to that, but instead of using a large language model and trying to arrest all this information about what emotions are, what they are in speech, what they are from the customer, and trying to overspend millions of dollars to try to come up with a solution which obviously isn't going to work anyway, we came up with a fixed model that kind of grounded in the golden truth. So you can either over engineer it, spend a lot of money and time, resources, in the end, you'll fail. We were down that pathway. We know where it ends. That's why we went this direction. Or you can find something that's a little bit closer to the actual phenomena and that way it's highly explicated, easy to explain, it can replicate, and you get consistent results. And how it works is when somebody speaks or types in text, we take those sentences and we run them through our partners. They do the audio processing for us and do audio signal processing and also turn it into a transcript for us. And then from there we analyze that with the Vern AI systems analysis. And each sentence by sentence, everything's on a lexical level. Then provide a score between like a 0 and 100 on a confidence slash intensity scale. Usually 51% or higher is significant for that emotion in that particular context. And then the more signals you add in, the higher the intensity. So I'll kind of give you an example. So if I said I you in a sentence that really doesn't tell me anything. I have two pronouns, I and you. If I say I love you, okay, I'm adding a little bit more information here. And incidentally, Rene, I would then trigger that as the love and joy signal. If I said I love you, Steve, you are an amazing man. You are courageous, you are honest and forthright, and I am really enjoying my time with you. That score would go all the way through the room. Right. We'd be up in the 90s or 100 percentage of confidence. And you can see how it would work because each signal we add to the emotional signal just increases the intensity and confidence that we have. That's what Vern provides is as somebody speaks, it gets that back to you in real time. So you can see what anger, sadness, fear, love and joy or humor that the person is trying to communicate versus

Speaker B: the last couple of years have just brought incredible advances in all areas of AI. And I guess along the way, I was just wondering who was going to tackle emotion. And you guys have done it, and I see it's working. I've tried it myself multiple times, and we'll probably give some examples here in the show, too. But let's talk about where you envision this being used. And I know the part of the answer is everywhere and the ideal answer. But I'm thinking, my emails, I always wonder, how does the recipient perceive my emails? I always wonder, even in texts when, um, I'm texting, maybe a client. I'm also starting to put myself in their shoes to say, well, how did I write that? How did I craft my words? And all of a sudden you start to get paranoid. And I wish was a system that could help me through that. It's like having another set of eyes on something before you hit send. So where do you see it getting used? Where are the industries that are clamoring for it now? And where can it go?

Speaker C: Oh, that's great, Steve. And email is a wonderful use case. Humans are emotional creatures. We just are. And humans use computers. Unfortunately, computers do not understand emotions. So like you're saying, whenever a human computer meets, kind of a use case for burn. But in the email is a great example because you want to understand the messaging and the information that's coming in. Imagine yourself looking at your inbox and you have an email come in. You go over to that browser tab and you look at your email inbox and you see subject line, the first little bit of the transcript. And then you see a metric which shows you how angry, sad, fearful, or joyful or humor is in the message itself. So you can actually, at a glance, understand which messages have the highest emotional impact, which messages are the most urgent, which ones need to be triaged first. So instantly on an inbox type of a scenario, you'd be able to see, okay, I need to address these things right away. Wow, this person's really angry. Maybe I need to get human resources involved. It's a Team member, for example, a lot of different ways in which you can use it inside inbox now, like composing an email as you go. You could compose and it could tell you how angry, sad, fearful, you know, any of the emotional that we pick up and how much so, so you can be able to tailor emails and, and yeah, truth be told, we use it for sales too. I mean when some people contact us we've been able to analyze, you know, when they have a little bit of fear, anxiety about not being able to get emotion recognition. Right. We know because Bryn knows.

Speaker B: Interesting. So I would imagine in any cases where, especially like in SaaS and cloud computing, I've always trained my teams for uh, companies I've run that because of SaaS and cloud, you're usually a two line email away from losing a customer. So Churn is a massive problem in determining your revenue stream as well as outpacing it by bringing on more new clients. Uh, I actually had a CEO from a software company tell me a couple of years back we were talking and I said, so how's your Churn rate? And he says it's pretty high. But luckily we're outpacing our Churn rate by bringing on new clients. And I just thought to myself that's a recipe for failure. The bottom is going to fall out at some point here when you're not bringing on as many new clients but your Churn rate is still high. You're going to be a uh, hurting company at some point here. But I'm thinking that it's obvious that it can operate in mental health situations. I mean I think that's kind of an obvious one. But I'm thinking that the audio streams of online gaming are monitored by humans all over the world because if you say something, if you bully someone, if you say the wrong thing, you get knocked out, they just basically say you're suspended or you're terminated off the platform. All the video conferencing is, yes, you've got some body language signals that are coming back to you, but you're not quickly understanding emotion. But then I also think about all of social media and all of social media and the good, the bad and the ugly. I mean it could be a cesspool too, we understand that. But there's monitoring that's going on by every social media company looking to understand who of their users are going against the terms and conditions of use on their platform and their kicking them out, suspending them or whatever each one of them does. They're monitoring all of that for various Things and I'm not going to get into the whether it's too much or not enough, but the monitoring is happening and I'm wondering if all of these platforms could be having more turbocharged power if they were just running it through your system. So now all of that being said, how does someone connect this up? Is it a big project? Is it? You go from one need to another. If one company wants to connect it to their CRM platform, their IVR System or their CCAs, whatever they're using, and another company wants to connect it into their video conferencing and what have you, how hard is this to get up and running? Or how do you serve it up, in other words?

Speaker C: Great question, Steve. Thanks again. Um, lot to unpack there, but we'll go ahead and start with the implementation. So unlike a lot of other AI systems, there is not a large on ramp, there is not a large need to ingest a lot of your past legacy data, uh, to come up and train a new model. That's not really how VERN works at all. It is a generalized model out of the box. You literally can get set up with it in less than two minutes. We've time people, even the slow folks can do it like me can do it under two minutes. But it's as easy as a registry for an API, getting the key and putting it in whatever type of array that you want to send. And you can start sending conversations through data, through anything that you need it to analyze immediately, and we'll get you back in real time. The API, obviously through cloud, is a vector in which anybody can add Vern in. But also we do have on, um, premises type of deployments as well, say like in a Docker image, those that are a little bit more sensitive to security and also, you know, to speak to that too, because obviously security and privacy is a huge deal. And especially when you're dealing with something like biometrics, like motion recognition. We've made sure that our systems HIPAA compliant, all of our software is suck too. So it's stateless and it doesn't actually end up stored on any of our systems. It works out of your system and goes right back in, in milliseconds in order for you to store the data and you're already compliant servers. So there's different ways we could do it. We even could probably do a chip set where we're talking a whole different type of deployment. But the closer it gets to the metal, the faster it is. Latency, obviously, in communications is a huge problem. That was one of the main goals that we had when we started was it has to be really fast and real time in order to have the value it needs. Interesting.

Speaker B: I find this thing to be so cool and I'm definitely giving it my cool stamp, which I haven't given many products in the last year. At least this season, I can't remember because it is just groundbreaking what you guys are doing. And any company, whether they're technology company looking to add emotion recognition, or they're a company that wants to use it for the aid or the help of, of their own customer service, customer success, there is just endless ways that you can introduce emotion. Now let's separate them a little bit. If I just have text based communications, I can use your engine. What if I have audio, can I also use your engine?

Speaker C: Yeah. We partnered with Cobalt Speech to have one of the world's first true audio and lingual motion engines. It's a multimodal motion recognition software that will do both the audio signals and also the lingual. There's a lot of myths out there and I'm sure you guys run across this quite a bit. And we do too. In our field, one of the myths that we hear all the time is like 94 of communication is non verbal. That actually comes from an episode of Seinfeld that says Cosmo Kramer, believe it or not. Yeah, famous episode. Google it, look it up. It's great. Kramer says that and then Jerry and George of course challenge him on it and he tries to communicate using facial expressions and grunts and groans. It is hysterical. We actually use this, and I give this sometimes with talks at universities. It's fun to watch people try to do this, but unfortunately it doesn't really work. And the reason is because the language holds the codex of our emotions and how we communicate. So while these signals are very important, they're extraneous, they're not necessarily not needed, but they add in additional information to what we already know the language is saying. You can go and look at a recent article in the last couple of years in Science where they actually prove that audio signals are only accurate at the lexical level at about 20%, 24%. Now granted, if you add them in multimodal and you add them with a lingual type of analysis like we have here at Vern, it goes up. So we found the same thing that the lingual does. Wonderful. And we get about 80% plus accuracy on all of our emotions. But when you add in the audio signals, that accuracy increases because you're adding Additional levels of logic and additional signal sets. So they work in concert really well together. And we've been able to show that they have a similar type of latency as our text only product.

Speaker B: I would imagine anything is subject to noise when it's an audio signal too. So hopefully it's as clean a signal as you can get. I'm just trying to think of what other areas. So you could do it without voice, you could do it without facial recognition. Is that an area that you're planning at some point in your roadmap, facial recognition too?

Speaker C: Yeah, actually we will be adding facial recognition at some point this year, maybe first quarter 25 to be a full multimodal experience. And that was one of the things we found working with Boise State University's GIM lab when we were working on the Army VR app for kids with autism, is that multimodal emotion detection complements one another. It's kind of like what we were talking about before. Having them all together just enables a system to be more human, like because it's replicating human processes and detecting emotions with your eyes, your ears and then your brain, because the language, what we use to code and label our whole world. So we're just Basically these giant LLMs running around, categorize and labeling our world as we experience it.

Speaker B: Since you brought up LLMs, let's talk about that for a little minute because I would imagine today OpenAI, as well as Google, as well as Meta, are all claiming that they can detect emotions also. And I'm not trying to bash anybody, but, um, I'm just wondering from an LLM perspective from a large language model, what should we be thinking about, what should we be concerned about and what are the right questions to ask if let's say we happen to partner with those companies anyway for other things and we say, hey, we want to do emotion detection too if we're not using vern. Help us be better buyers in general, if you will.

Speaker C: Oh, absolutely. That's wonderful. LLMs, by the way, I love. We use them all the time. There's conversational design that we do with bots for clients when we build in Vern that actually is very helpful. It rounds out the experience and it makes it more human like and more flexible. So LLMs are crazy amazing technology and I'm not here at all in any way, shape or form to bash them. But as with any technology, there is weaknesses and usually the weakness is in the methodology. Unfortunately, this is also true with LLMs. Part of the what makes them magic is also what Makes them terrible at emotion recognizing. So you've used autofill, right Steve, you've written something out recently and you've had suggestions on how to finish that, or you're typing in a search in a search window and it automatically is giving you some recommendations. Well that's what's called an autofill program. And basically what an LLM is, layman's terms, it's an autofill on steroids. It gives you the next answer based off of the highest probability at the time. So every time that you ask an LLM a question, you're likely going to get a slightly different answer. So if I'm using something like an LLM to do measuring anything like emotion, it's essential that it actually is consistent and accurate with the same stimulus every single time. Otherwise there's no external validity. I can't actually see this out in the real world. It doesn't pass the SNP test. But unfortunately with LLMs, because they source a wide variety of source material, Internet books, academic papers, they come across a lot of the discombobulation within psychology when you're looking at all of the different models M so the LLM doesn't know which model to choose, the 5, 7, 9, 13, 27 or 40. So oftentimes it will pick the best it thinks at the time at the very second that you ask the question or it will go with more of a consensus building. But these all have conflicts. Whether you're talking a cognitive model or a physiological model, they all have conflicts in what they consider these emotions. So your LLM is always going to have conflicts with the results. And we asked this of ChatGPT4, of Claude Llama, Coors GPT Hume just had a recent demo where they claim to detect emotions and they all do the same thing. It's essential to have a, ah, consistent measurement every time. If you're going to be using it in mental health like we do their partner evermore, then you have to be accurate because it's a scientific measurement tool, it's an instrument, it's inaccurate, then you really don't know what you have. And unfortunately every time that we add the same stimulus to these LLM systems, we get a different answer every time. And it's kind of striking to see that you give the same thing and get a different result and realize that this could actually hurt people.

Speaker B: So I would imagine if you're testing other systems, you probably want to do the same thing multiple times, maybe say the same thing different ways. But uh, see if you're getting the same scores because I would imagine if I'm putting a developer's hat on right now, if I'm connecting any other method of detecting emotions, I've got to take whatever's being passed through to me for signals and then I've got to interpret that in code and then be able to send back to a screen, back to a soft phone, back to a CRM system. I've got to send back kind of an amalgamation of where things are at that particular moment in time. And if I don't get the same answers, I can't do that. I have to just basically say it's the answer they've given us, it's the answer the LLM has given us. That is not good enough. Because an LLM is not the de facto authority on information. It should be those of us that can interpret properly and are expert in certain areas. I'll tell you, this has just been a roller coaster ride, Craig, and I really appreciate you taking the time. I know how busy you are and we've been after you for a while to try to get you on here, but I really appreciate this because this area is going to just explode out there. And of course I'm sure you're going to be getting a lot of calls, get a lot of people that are looking to try things out. And you said it was easy. It's like two minutes to get access to the APIs. Do you have some kind of a trial period or is there some low cost way that a developer or a, uh, technology company can try this and see how important this is going to be to their future?

Speaker C: Yes, absolutely. We have a free trial period, goes through the 14 or 30 days you mentioned. This podcast will give you the full 30 days, no matter what size your organization. And this gives you an opportunity to kind of get in there, test it out, work with us a little bit, uh, see how everything goes, and then apply best practices.

Speaker B: This is great. Craig, how can people get in touch with you? What's the best way for people to get in touch and to learn more?

Speaker C: You can just email me directly. Folks, I have no problem with that. It's craigurnai.com the website is vernai.com and I'm available on LinkedIn and other platforms.

Speaker B: Great. We'll put those in the show Notes. Thanks again for taking the time and enlightening us with a view to the future. Thanks again, Craig.

Speaker C: Thank you, Steve.

Speaker B: Well, that's another episode of the Science of cx, everyone. I hope you're as excited about this technology and this area of business that that just sounds like you have to look at it no matter what. Take a look at it and see if it's something that can help your business moving forward. If nothing more, you can predict better your churn rates, you can understand your customers better, and I think that's pretty darn good. So once again, until we meet again, please stay safe, stay healthy and do take care everyone. Bye bye.

Speaker A: You've been listening to the Science of CX My name is Steve Pappas. I, uh, really hope you've enjoyed this episode and if you have, the highest compliment that you can give us is to subscribe, rate and review the Science of cx. Thanks and we'll see you in the next episode.

Speaker B: Uh,

Speaker A: finding one place to see all customer experience related tools and technology has been difficult until now. We just built it. Get ready for a Science of CX Original Customer experience technology has been helping to drive businesses by giving them insights into better methods to engage and delight their customers for some time now. But if you're looking for CX tech, you have to search far and wide to understand the whole landscape. CX Stash is your simple one stop directory of all the great CX related technology you need. It breaks down all CX by collections like analytics, CRM, Journey Mapping, Voice of the Customer, UX Customer Support, and more. It's free to create an account and use no advertising cluttering up your experience. Just one place to find all the great CX tech. Sign up today at www.cxstach.com.