Unresolved.cx - The customer experience signal AI doesn't know it's missing - Robert Cabral - Runway

Brett Rush: Welcome to Unresolved, the podcast for customer experience Leaders by Customer Experience Leaders. In this podcast, we're going to focus on how different companies view and measure their customer experiences, how they handle Churn, and then we'll dive deeper into the unresolved issues that they're facing today. We're specifically calling this podcast Unresolved because so many CX leaders are testing with AI. They're rebuilding with AI as the foundation, and every one of us have to reinvent every everything we've ever known about customer experience, systems and processes. You're not alone. So let's do this journey together. Today we have Robert Cabral, the head of customer experience for Runway, an applied AI research company that builds video and world models. And Runway is used by filmmakers, studios and creative teams to make content with AI. Robert, welcome to Unresolved.

Robert Cabral: Thank you. Really excited to be here.

Brett Rush: I am so eager to dive into your story for. For those that haven't heard of Runway, they were early in the creative AI video journey when people still didn't even know how to write prompts. And I remember two years ago, back in 2024, uh, I took a picture of a marathon, and it was like a head on shot of a marathon, and there was a person in a chicken suit. I explicitly wanted to see what would Runway do with that detail of the feathers? And I was honestly blown away. And getting. This was two years ago. I can't even imagine now, but like, two years ago, it was getting the detail of that chicken running and the feathers running. And, uh, it just blew me away. And, uh, and it's just, it's amazing what the product was and how it has evolved, I would say, past any other video creation model that, uh, that's out there today.

Robert Cabral: Yeah, I mean, I would agree. Um, I might be a little bit biased, but the thing that's really cool about Runway is that it is a research lab for us. So we're building our own models, we're doing the research, we're fine tuning things. So, two years ago, honestly, if you were to look at the stuff today versus two years ago, you'd probably laugh at how much things have improved. So, I mean, I would love to take a stab at that chicken suit guy and run him through Runway one more time. Um, working in AI, it's all about change and adapting as a business. But that also applies to the entire customer Experience team and being extremely nimble when it comes to that.

Brett Rush: Before we jump in too deep, I'll start with some grounding questions and that, uh, way the audience can get to know you and uh, we'll get to know how your team is using AI, what worked, what hasn't worked, and then I want to spend some time on what's unresolved. So just to kind of open things up. Tell me about the team. What's the size, the structure? What does your team actually own?

Robert Cabral: Yeah, so we recently rebranded the team from customer support to customer experience. And the reason for that, even though it is a pretty small team, is that we've identified, we broadened the scope of what we do and a lot of it is because of the AI that we've built out in the infrastructure. So we have consumer support, enterprise support, we have customer education, and most recently we have a scaled success division which is basically any enterprise accounts under a certain threshold that get a different level of hands on account management and support than maybe some of our other customers. But we want to make sure that we scale that as the business grows. So like I said, it is a pretty small team and we're being very intentional about that. And I know some, some companies and, and some leaders try to boast about, you know, my team is like 30 people or 500 people. I've actually never had a huge team and I like it that way. Doing the most with, with the people that we have. Um, and AIs really enabled us to, I think in terms of headcount, we haven't really grown the team in the entire year and a half that, that I've been here. We, we obviously started off with our ticketing system and I, despite being part of an AI company, we didn't jump right into AI support. Part of it was really understanding what, what are we doing well right now, how can we improve that, how can we double down on that? And then if we do move forward with AI, what's, what sort of infrastructure do we do? We need to set up before that and documentation and education is a big part of that. So that's why we have one person, Hayley Mills, best in the business, who, who builds out our documentation, our guides, super, uh, thorough. And that has been sort of what's really helped us stand out in terms of the AI support, uh, experience. In my opinion.

Brett Rush: That's great. I mean you're, you're leaning right into my next question which is uh, when you're evaluating how to build an operation for customer experience, how do you think about what belongs in your tech stack? I mean you're talking about uh, Hayley is like in documentation. So, and even you're talking about not scaling the team so much, but really you're scaling their skill set.

Robert Cabral: Yeah. So I think before, before even the tech stack, just hiring the right people is super important.

Brett Rush: Absolutely.

Robert Cabral: Everyone on my team has some sort of creative background. And you know, whether that's, that could be working with creative tools like Runway, but we also have someone on the team that has a dance background and that just, that appreciation for the arts, I think is, helps put ourselves in the customer's shoes, what they appreciate. You know, when, whenever you talk about art versus AI, there's some conflicting views there. And I see Runway as really a, uh, creative tool. And that's what it is. It's a tool, it's not a creative itself. Um, and in terms of the technology that we use, it's also a means to, to what we need to achieve. So the way that I approach it, the way that I think about it, is going back to the principles, what is it that we want to achieve? What is our ultimate goal? And breaking down what are those smaller, more tactical things, more and broader strategic things that we need from an AI company. And one thing that's a little bit less tangible, at least in the review process, is working with vendors, with companies that are as forward thinking as your company, as quick as your company. Uh, you know, we're making updates, uh, sometimes on a daily basis. And that impacts what our needs are from our vendors. And we, we want to be able to work with an organization that grows with us basically. So that's a very broad way to answer your question. Uh, but taking a little bit of a step back, one thing that I found really important the second time around going through the vendor evaluation was really bringing in someone from our engineering team in the entire process and understanding, you know, we are launching these things on a daily basis. What, what do we need from a technical standpoint? I'm not a technical person, so there is only so much that I can evaluate for on my own. So it's really important whether you're a technology company like Runway or maybe another type of company to bring in the right people to build out what those minimum requirements are.

Brett Rush: So I'm working with a company that they, they're at a crossroads of how do they connect everything together? And, and I said, I need to talk to the cto. And initially, again, rewind. A few years back, that wasn't really something that support or success or experience needed to do. But now with, with the state of like MCPS and everything connecting and data being able to be shared, being able to be accessed. Uh, it just, it makes total sense now that the CTO needs to be connected to customer experience and uh, product needs to be better connected to customer experience so that you can get that information to all the different departments. We're not in silos anymore. We can't be in silos anymore. And if we are, we're, we're not going to be able to grow the company.

Robert Cabral: The way to really position it internally I think is you're taking up an engineer's resource, which, which is time and resources, which is very valuable. Yeah, in the short term is going to be a bit of a lift on their end. But I think longer term the value is they're spending less time trying to get this system built that maybe wasn't the best system for you to begin with. Um, so, so yeah, totally agree on just kind of not thinking of support or experience as these silos. In fact, most teams nowadays that I speak to are building their own tools and using these MCPs to build out their own automations. And that line is starting to blur a little bit between product and customer experience, in my opinion.

Brett Rush: Yeah, and you had mentioned uh, before about not jumping right into AI and thinking about the human processes. So that said, like thinking about the state of the current customer base for Runway, do you feel like customers expect AI support because Runway produces AI generated content? Or does it seem like they expect a human interaction because it's an AI driven product?

Robert Cabral: Our customers are more creatives first. They absolutely use technology, they use AI, but they are creatives first. And at the end of the day what they want is bringing their ideas to life end to end and ideally with, with minimal support without having to reach out to anyone. So I wouldn't necessarily say they expect Runway to have AI support. In fact we have a, ah, discord community that is very strong. Uh, I'll give a shout out to Timothy Highley who runs the community and does a really good job there. So I would actually say that type of human support is expected from the creative community. And, and when we do roll out AI support, the expectation is not necessarily that it's there, but the expectation is that it's top notch. Like it's gotta be, we're in AI company, so the bar is much higher when we roll out an AI platform to support our customers. So that, that's probably the distinction there in terms of expectations.

Brett Rush: Yeah, I used to work at Frame IO and we were starting out by creatives as well. There was really no reason for folks to get into, into Frame IO Unless they were. They were creating. Creating videos. And I started hiring people from production. Like people that, uh, not only they were. They were in production using the cameras, they understood how to manage a red camera, but they were in post production. They had an expertise in Premiere or Resolve or something along those lines as well. So, yeah, hiring creatives is absolutely crucial. When you're in a creative space, you can teach people how to be customer support. And also if you're in a creative space, you have the mindset of troubleshooting already. Like, how can I figure things out? How can I learn more? How can I expand my. My horizon, my thought process? So thinking about Discord, it's so funny that you bring that up because so many companies, I see them, they're using Discord. And, uh, the only reason why I say that's funny is because I think it was 15 years ago. Forums were a really big thing. Before Reddit actually blew up. Every company had their own individual forum. Uh, I also offered Skype support, so, like, let me keep dating myself. We were doing phone support, we were doing video Skype support. We had the forum, we had email. Chat wasn't a thing back then. So it's just like. It's crazy how kind of like, forum just kind of came right back in the way that it used to be 15 years ago. A really strong thing for support.

Robert Cabral: Whenever you have a tool like Runway, and it's really valuable for people to learn from each other, but also from, you know, Runway experts in our case, um, that's something that you don't want to lose, and I'm personally really grateful for that.

Brett Rush: Yeah, I feel like customers, users, they could. They could open up a little bit more in a forum than they would if they were talking to an AI chat bot or even, even through a chat to a human. Like, they kind of. They kind of bring in more flavor, more taste to, uh, to the conversation whenever it comes to Discord. Um, so thinking about AI, let's. Let's transition over. You told me before we started that you launched AI support. It's been quite successful. So tell me what that means in your operation. Like, what did you build? What does it handle? Uh, what's the before and after actually look like?

Robert Cabral: Yeah, so the main reason we built it out was to give our customers faster responses with the same accuracy, ideally, and really just help reduce the amount of interruption that they had throughout the creative process. If you do have a question, you have to wait. Even if you have to wait like 30 minutes or 15 minutes, it's it disrupts the process and you're no longer in that creative mindset. And that's what we wanted to prevent. As much as possible, we're getting super fast responses out, but we could see that there was a gap.

Brett Rush: Yeah.

Robert Cabral: So that improving that, that entire customer experience, really helping customers use the product, that's, that's the ultimate goal there. Um, so documentation was, was the first piece, if embedding that into the platform before we even rolled out AI was really ideal for us. Um, but then as we started to grow, one of the things that shifted is more of a focus on enterprise support. So we have consumers and enterprise support, and it's a different level of support, different type of questions. And that was another thing that we wanted to address. How do we make sure that we provide excellent support for both cohorts and AI seemed like the good solution for that, where we can leverage this amazing documentation that we have and have it address those very simple questions like how do I reset my password? What model should I use for this and that? And then our team became more of like the level two, level three, uh, support that could answer all the questions that are a little bit more nuanced, a little bit more complex, may require some of that industry knowledge that maybe the chatbot doesn't have. So to answer your question again, I think it's really scaling the business as we grow both for consumer and enterprise. And also number two, making sure that that customer experience remains as uninterrupted as possible.

Brett Rush: Yeah. So you had to, you had to rewrite the documentation so that, uh, AI could find the right information and provide the answers to your customers. So the AI chatbot really handled a lot of the informational questions. Are there any actionable steps that you're providing it to either, uh, handle refunds or handle anything like that?

Robert Cabral: Yeah, actually, exactly that. Refunds and billing questions are a bit of a pain point that you have anytime you're working with consumers, um, especially when most businesses have an auto renewal system. But most customers, even myself included, may forget like, okay, I need to cancel this, I'm not using it anymore.

Brett Rush: Right.

Robert Cabral: So we have built in mechanisms and that's, that's where that engineering part partnership comes into play to understand what is this person's usage, when did their account actually renew and what's, what's the reason for cancellation? So that AI can make a decision, should this person get a refund or not? Should some, should a human actually evaluate this versus the AI? And we rolled that out and it's been pretty Successful. Um, one of the things that our leadership team was concerned about was how do we make sure customers don't abuse this? And that's where we built out the mechanisms on our end. So an AI, AI support enables us to do that, but it's really following the strict guidelines that we have in place. And it's more hard coded, uh, decisions versus things that the AI can hallucinate. So that's how we. Those are the sort of guardrails that we have against that.

Brett Rush: That's good. Yeah, those guardrails are definitely necessary. There was a company I was, I was talking to that actually did not think about putting those guardrails up. And every single month these customers figured out that they could just continually ask for a refund. And AI is like, sure, there you go. And they could just keep on using the product. And uh, they realized that I think six months in that there was a lot of customers just, they're like, what, what is this number going up so high? And our user count is not going down.

Robert Cabral: So yeah, don't get me wrong, I, I'm a little paranoid about that too. And every, every week or so I'm like checking our reporting, making sure there's no, uh, spikes. I think everyone has to do that anyway. That technology can go wrong.

Brett Rush: It's so funny as being in customer experience, you don't have to think about what are the people's best intentions. You got to think about what are their absolute worst intentions and how are they going to game the system and how are we going to mitigate that, fix it and uh, make sure that, uh, you think about every wrong possible scenario to solve for. It's wild that not, uh, a lot of people have to think that way, but support and experience folks definitely have to think like that.

Robert Cabral: There's always that one person or group of people that finds a way to break the system. So. Oh yeah, I hope you're not listening to this podcast if that's you.

Brett Rush: Definitely. So thinking about things that could go wrong, like can you share with us a moment where maybe you failed forward with AI? Uh, you learned quick, you fixed it, uh, or you changed something.

Robert Cabral: Yeah. So we were prioritizing for making sure that AI can handle as many questions as possible. And one thing that I think we could have done a better job at, which we did quickly adjust on, was that handoff process. What does it actually look like when the customer goes from AI support to human support? And how do we make sure that there's no frustration in that handoff process? I Always think back to phone support. Even now with all the technology that we have in place, you have situations where you're giving your order number and then you're handed off to somebody else and you have to repeat it all over again. And then the cycle repeats maybe two, three times. It's a huge pain. And unfortunately that's kind of what was happening with, with our system. In AI, we were making that handoff and we had our team having to go back into the AI system to check for that context or ask questions that were already answered by the customer during that AI chat. We quickly identified that, I think first week or so it was pretty, pretty obvious to what was going on there. And we built in systems to really make sure that we optimize the AI, not just for having those conversations, but for aggregating the information that we're getting, submitting that information as an internal note to our team, including attachments that the customer may have submitted, and really making it very easy for the team to just quickly scan the summary instead of the entire conversation to understand what's going on and really speed up the process. And that has vastly improved the customer experience. Uh, the perception of AI versus human support as well.

Brett Rush: Yeah. So good. Yeah, I mean it could be going back and forth 11 rounds of Ah, replies and you want to spend all that time reading paragraphs and uh, and links and oh wait, that's not what, what you were asking for. Let's get down to the real nitty gritty. Yeah, that summary is absolutely crucial for sure.

Robert Cabral: Right?

Brett Rush: Yeah. So, thinking about signals, I want to kind of shift things to what you said matters most. You told me that what is unresolved for you is uh, scaling without losing that signal. So now that AI handles most of the volume or more of the volume, uh, what's the risk to that signal? Do you feel like there are signals that you're concerned about losing?

Robert Cabral: For sure. And I, I think this is an ongoing thought exercise, basically going through conversations manually. I, I use discord as a signal now as well to understand and compare what are we seeing in the community, what are we seeing in our AI, uh, conversations. Uh, because AI in general thinks it's really smart, it knows all the answers, it knows exactly what it needs to do and it, it just doesn't. And I, I, I feel like if you're rolling out any sort of AI, you have to understand that, that it is a process where you have to iterate, you have to understand, you have to look at the data, um, if it's having A conversation with a customer and just going back and forth and being very confident in giving these resources, having no context on what bugs we have in the system. Or, you know, maybe this requires a different approach. Those are the types of signals that I'm looking for. And there are systems in place where you can capture those signals to a certain extent. Um, but there's so much nuance in that as well. There's so much nuance in the signals in general when you're having a conversation with a person. When everything came through human support, the team would just have conversations like, what are you, what are you all seeing today? Um, you know, everyone's complaining about generations taking longer than, than usual. Oh, yeah, I saw a couple of tickets about that too. Whereas with the AI support, it may default to the typical, you know, you're having generation issues. It's probably your browser or something like that, and it may respond to 50 customers that way before, uh, anyone realizes maybe there is actually an issue going on there. So that's something that I think we need to get better at in our team. But I think across the entire industry, as. And I think we're getting there, uh, AI is constantly improving, is capturing those signals. And if you're not able to doing the hard work of just going through things manually, talking to customers yourself, um, making sure that your team is extra vigilant about the conversations that they're having, uh, versus just going through the motions. So there's, there's only so much technology can do, but at the same time, there's a lot that it can do to, to address those types of issues.

Brett Rush: Yeah, yeah. I mean, it loses that gut feeling, right? It's, it loses that human gut feeling or even like, hmm, something's off here.

Robert Cabral: Yeah, right.

Brett Rush: This is the third conversation I've had about that. Maybe it's something else. Nope. AI is like, this is the answer.

Robert Cabral: Yeah, exactly.

Brett Rush: Yeah. So even to that point, like an unresolved. And maybe this is solved, uh, and somebody doesn't know, like, uh, I don't know about it yet. Maybe like there's a spike in an issue and it tells the support system, hey, this is what's going on. You might see this, right? It might be great to have that kind of connection. I'm not sure that any, any product is there yet, but that'd be great to have that intuition, hey, seeing an issue over here and it just lets it know and it's like, oh, okay, maybe I should think about a good gut feeling response to this or even notify Folks and customer experience. This is what's going on. Uh, and there's some, some kind of a red flag even either through like a Slack or Teams or Discord or something along those lines. Like, it'd be great to have all of the signals from all of the platforms working together and being like, hey, red flag over here. Something going on.

Robert Cabral: Yeah, well, I am kind of working on something myself. We've got. Everyone's got, uh, an LLM integration and account in companies nowadays, and we get a lot of leeway and are able to just build things out. Everyone on my team builds things out. I am jumping on the bandwagon myself and building things out. You know, it's easy to put the excuse like, I don't have time for this, but you really have to make the time for things like this is super important to capture those signals. And if it doesn't exist, you can build it yourself. There's. There's really not an excuse nowadays.

Brett Rush: Absolutely. Yeah. So I have one last question for you, Robert, and it's if you could describe the ideal customer experience and Runway, and maybe it doesn't exist today, but it's like, think forward motion. How do you want the customer's experience to actually look like, even though, like, maybe the technology is not there today? Like, what would that workflow look like? Would it be every action identified with defined solutions to be highly autonomous, which it doesn't feel like that for, for like the, the creatives? Or would it look like more customer sentiment and, and signals being able to be detected that require that common sense and a human touch to, to respond? Like, what does that customer experience look like for Runway in a future state?

Robert Cabral: I've said this in general, but I think the best customer experience, no matter where you are, is where you're just doing what you need to do and you're done and you leave. That's it. You don't have to worry about how do I contact the support team or what kind of documentation can I find. The product is just so intuitive that, um, you really only have to reach out to, to the team for those super nuanced, uh, complex questions that you have. And maybe that's through proactive, some sort of proactive support where you're working very closely with AI and with your product team to identify, hey, it looks like this customer is stuck in, in this page. They've tried to generate something like four different times. Maybe we should offer a suggestion based on what prompt they're using and, um, how long they've been going at it for, uh, we may have enough documentation and resources to give a proactive suggestion or help uh, them on the right path, maybe suggest a different model. So that's how I consider that's the best experience in my opinion. I think it's more of a matter of when we'll get there. I do think it's pretty possible, but um, that's kind of how I see it. Like you have a support partner that can just jump in and help you out whenever you need it before you even think of asking.

Brett Rush: Yeah, yeah, you're on the right track because I've been thinking about the same exact thing. If you think about Photoshop and uh, anytime somebody thinks about Photoshop they see a hundred plus buttons and options like you really have to become an expert before you could even do 10% in the product. And if you thought about, if you could just explain or prompt. This is what I'm trying to do. Here's a couple of options of uh, images or something. Can you layer things together? And then my thought process is that Photoshop would kind of like, here's the history stamp and you could go back to that history and, and change it so you're not actually, it's, you're learning how to use the product through prompt versus having to figure out what every button actually does. And maybe it's even like adding gradients the way that it needs to be added. Uh, I was trying to turn a picture into a stamp, a rubber stamp, and it took me four hours in Photoshop the other day and I ended up hand drawing it because I couldn't figure out an option to make it look like a rubber stamp.

Robert Cabral: So we need to have a Runway session together.

Brett Rush: Absolutely. Yeah. Um, so, yeah, like I think you're on the right track. I think that uh, product is going to be so integrated into the customer experience that it's going to be like, wait, you did this? And uh, you did it again. So what are we trying to achieve? And let's do that together. I really feel like that's where the product, products are starting to head now.

Robert Cabral: Yeah. And I can't help but think of the annoying paperclip that Windows had. It was, I think I remember it being helpful sometimes. Um, if we were able to do something like that back then, I mean there's no reason why we can't build out something like, like this experience.

Brett Rush: Yeah, yeah. You know, I, I use Notion and I think of Notion's, uh, little icon as being clippy for me, that's my clip. Robert, I appreciate you taking the time to speak with me today. And I love that you're sharing what you're. What's actually in progress, not just what's solved. Uh, and that distinction that you made between what AI can answer, what. What AI should answer, it's something that this audience is thinking about. And I, And I love even the product of thought about what customer experience should look like within a product. Um, I think there's so many folks that are in the same boat. We just got to figure out if the tooling is there yet or if we're just not doing something right. And, uh, hopefully we can bring in some answers. You can connect with Robert on LinkedIn and that link is in the description. Uh, Robert, is there anything else, uh, you want to, you want to share? Is there anywhere else people can find you?

Robert Cabral: Uh, yeah. I mean, first of all, thanks for having me. Really hope that this was helpful for anyone, uh, listening. I do also have a, uh, beyond the Queue, uh, blog that I try to post to on a weekly basis if anyone's interested.

Brett Rush: Awesome. That link's going to be in the description as well. Um, that wraps up today's episode. Subscribe to hear more incredible stories in the era of unresolved. Have a great day and a productive week.