On Device AI vs Cloud AI, How Sensory Built 30+ Years of Voice Innovation with Todd Mozer

Speaker A: Built to Scale with Reem Benchar. This is Built to Scale, the podcast for B2B founders who are playing the long game. I'm your host, Reem Benchar, and every week we are exploring the mechanics of sustainable growth with the founders and executives who are defining the industry. No fluff, no growth hacking gimmicks, just the blueprints for building a solid and scalable tech company. Welcome to the show. Today we have a special guest joining us, Todd Moser, founder and CEO at Sensory. Todd, thank you so much for joining us today.

Speaker B: Thank you, Reem. I appreciate your offering me to be here.

Speaker A: Well, I'm really glad to have you here. I think this is going to be a great conversation. So tell us more about yourself and Sensory.

Speaker B: Sure. Sensory is my third startup and the first one went public, the second one, um, was acquired. And Sensory has been a, uh, long run venture. I've been running it for 31 years now. And what Sensory does is develops unique on device technologies, AI technologies that we license. And even though there's kind of a new wave of, uh, voice AI, we've been doing neural nets m with speech recognition for 31 years. So we do biometrics, we do wake words, we do speech to text. We have a variety of technologies that we license mostly to product manufacturers. So we're a B2B business.

Speaker A: Amazing. Thank you for the intro on Sensory. I mean, I was impressed when we talked the first time you said you've been doing this for 30 years. You know, 30 years is a really long time. What do you think has kept Sensory durable through multiple tech waves?

Speaker B: Well, uh, there's a lot of ways to answer that question. I mean, durability for a business in general is based on profitability. Companies go under when they run out of money and can't raise money. So we've, uh, certainly been profitable in more years than we've lost the money. And so in general we're a profitable company and that gives us a lot of durability. But a deeper answer to your question might be the innovation and the people at Sensory that create those innovations. We've been able. We have, uh, got an awesome team that really hasn't turned over much. I think our average tenure is like 15 or 20 years at sensory. So people join and stay. And those people are what makes Sensory succeed and develop new technologies that always stay on the cutting edge.

Speaker A: Great. Well, for someone who is new to the space, what does on device AI really mean and where does it beat? Cloud only approaches.

Speaker B: Sure. So there's a great Reason to have cloud. And there's a great reason to have on device. On device is better if you're not cloud connected. So on some devices, like say a car for example, you're driving around, you're not always going to have a great cloud connection. So you want things to always be available, always work and be accessible. Um, privacy is another reason people like on device because then the data that's uh, being used never goes off to some third party which could get hacked or could be selling your, your information. Power consumption, we're in a lot of wearables these days and power consumption is a big deal. So being on device, low power is really, really important. There's a whole lot of things I could keep going, but um, there's a variety of reasons for on device. The reasons for cloud is that you get better models in general. The bigger the better in most cases. And if you want to use a large language model, it's going to perform much more broadly and better in a, ah, giant formation in the cloud from super heavy processors. Now there, there are advantages of moving those large language models onto device. For, for small language models you can avoid hallucinations and build an expertise in a very specific domain by doing that. But you lose some of the general knowledge that you have from a cloud

Speaker A: ll that makes sense. So you had mentioned before the hybrid model, right? The device plus cloud. What do you think are the biggest practical benefits? Bandwidth, latency, privacy and relatable. Relatable, sure.

Speaker B: Uh, it goes back to kind of the question I was just answering. You get the best of both worlds. So you want to do, for example, you start with a wake word. You want the wake word to be on device. You know, that's the uh, Alexa trigger that wakes it up, for example. And that needs to be low power. It needs to be always on, always listening. You want it to be private. You don't want your kind of general conversations going up to the cloud for analysis. Following the lake word, you're going to have some kind of command or question or interaction. Now that can be speech to text on device, which offers a real advantage because then you're sending a lot less information to the cloud. Speech data, vocal data, is uh, a lot more bandwidth heavy orders of magnitude bigger than sending text. So if you can actually do the translation on device and send the text to the cloud, you reduce the bandwidth, you reduce the cost, you reduce the latency. But it's in the cloud that you get these amazing models that can handle any language and, you know, really understand what you're saying better and better.

Speaker A: Would it be possible just for our audience and myself to, to understand a little bit better what some use cases for your, your technology could be like? Right. I'd love to have maybe a few examples so we can kind of wrap our head around how that looks like in real life technology and usage.

Speaker B: Sure. Well, uh, we've shipped in about 3 billion units over the years. There's a whole lot of hundreds of different products have used our technologies. So one of my favorites are GoPro cameras. You know, that's a really nice device where people have them mounted on their helmets and things like that. And you don't really want to fool around with your camera while you're skiing. Your hands are busy, your eyes are busy and so you can say, hey, GoPro, take a picture, take a video. So, so that's something that happens completely on device. We have other hybrid models where we've done wake words and biometrics that get then sent to the cloud. We've been in a lot of mobile phones over the years. We're in a lot of wearable devices today. Um, and the idea there is you can wake it up, you can identify who the user is and send that information to the cloud for analysis, whether it's a watch or some glasses or um, a variety of different wearable devices. We're in a lot of medical products today too. Medical products. And there's HIPAA privacy laws that um, that enforce more usage on device so that everything isn't being sent off to the cloud and to uh, data centers that are outside the control of the hospitals. So we get used in a lot of products like that.

Speaker A: That makes sense. And I feel like there's so many more use cases I could think of now for the use of the technology that you have developed at Sensory. So that is really interesting and I do feel like with of AI that is happening right now, we're just thinking purely in cloud based. But um, you know, when we, when we look at the day to day usage of products, that's not always relevant, if that makes sense.

Speaker B: That's right. No, that makes a lot of sense. I mean the automotive industry is probably the best example of where you really want a hybrid combination. And the whole auto industry is going through, you know, a huge shift right now that cars are becoming autonomous and that's expected to lead to sort of shared vehicles that everybody uses and the expectation of an interaction by voice where a car isn't just going to get you from one place to another, but it's going to entertain you and you're going to be able to order your pizza to pick up at the next stop and all these things. And having that combination of on device in cloud so you get the privacy advantages, but you get the power of the cloud is super important. And what we're seeing in a lot of cars now is they use Sensory on device and then they'll use for example Alexa in the cloud. We're in the BMW cars right now with Alexa, uh, services, but Sensory on device. And so that's kind of a fun combination that we're seeing.

Speaker A: Definitely. So Sensory went from essentially from chips to software. When that happened or before that happened, what prompted the shift and what did it unlock for growth and distribution?

Speaker B: Sure. Um, question. I mean we, we've always been a software company so it was really more a shift from software and chips to just software. And when I started the company I named it Sensory Circuits. And within a few months of starting it, I knew we were going to be a software company so I dropped the circuits off of it. But we did design our, our first chip really because there wasn't much out there that fit our needs at the time. And so we designed a little 8 bit microcontroller with 64 kilobytes of ROM on it. And, and we could sell a chip that could talk and could hear and could do brain functions processing for under four bucks, um, which was unheard of at the time. It was really the first successful um, speech recognition chip. And um, after a while we realized that our real strength was in the software. And um, when it came time to do a new chip, chip designs are very, very expensive. We decided let's instead let's move to other people's platforms. And so as a result we now support dozens and dozens of different platforms. Um, Qualcomm just announced their new chip. It's their elite product line for elite wear, so wearables with the uh, Qualcomm Hexagon and Low Power Island. And Sensory is one of the partners that's supporting that platform right from day one. And it's exciting to be on these kind of bigger products that are being marketed out there to the world. So going from chips and software to just software really has brought us into a lot of partnerships with a lot of chip companies. We support all the IP platforms like Cadence and arm and then we work with a lot of chip companies like STMicro and a uh, whole wide range of chips.

Speaker A: That's exciting. Definitely some good stuff you're working on and projects So I appreciate you sharing that with us. So what we're seeing right now is that voice is exploding again with LLMs. What do you think is changing in the market and what are teams getting wrong right now?

Speaker B: Well, everything's changing in the markets. Um, LLMs are getting better, faster and faster. There's new releases coming out now every month and pretty soon it's going to be every week. And we're seeing rapid, rapid improvement and the ability to really have them be really functional, not just as interactive things that you talk to, but as agents that go out and do things for you. You know, the whole Claude bot and went through a bunch of names, but that little creature, it's given the ability to identify your LLM and you know, it runs, runs on your computer and so it stays somewhat private. The LLM isn't private, but that really gives you the ability to do so many things. And I'm not sure companies are getting it wrong, but I think they're probably not moving towards it fast enough. We had, um, an internal conference. I mean, we're an AI, a neural net company and we're too slow. So I'm always pushing our company to move to the best LLM, to use coding agents and to use agents for everything. We had a, we have some job openings at Sensory right now and we were talking about, okay, what is going to be our process when the, um, resumes come in? Should it go to hr? Should we have the product manager review it? And there was this concern about, well, it's going to take a lot of time for the product manager and maybe HR can do a first pass. And I was like, no, no, no, let's, let's hook up an agent to look at the job descriptions, compare it with the resumes that we get in and have it build a table and rank all the incoming resumes. And, and I did that last night in a couple of hours. I mean it was, and I didn't have any experience doing it. I actually don't have a Unix or a Mac based system, so I couldn't use the, the cloudbot type of approach. But, um, you know, I, I could use Google to do it. There was a way to do it and Gemini helped me set it up. You know, when it didn't work the first time I asked Gemini, you know, how to fix it. And it was this interactive process that's, that's amazing.

Speaker A: It really is. It's incredible. I mean, there's so many exciting things and things are moving very, very quickly, but our capabilities are just 10xing as we're speaking, because we have these incredible tools that are essentially accessible to everyone, even if you don't have any coding, uh, the right tools in order to make that happen. I think one of our last questions here is the following. So when big companies want to ship fast, why do they partner versus built in house? And what should buyers look for in an embedded AI vendor?

Speaker B: Sure. Well, there's a lot to unpack there. So let me, let me just start with, um, sort of the decision. It's really, do they want to make it, do they want to buy it, or do they want to license it? So we certainly get approached by companies that are looking at, at our technology and they say, hey, let's just acquire them. And some companies get acquired because the bigger companies would rather, uh, purchase the company rather than license or make it in house. The drawback, you know, most people can throw resources at things, but it's not that easy to do some of the stuff that we're doing. You know, we've got thousands of man years behind our technologies because we started with neural nets and we started with small platform. We really, um, have built an amazing code base that just works. And it's code that's not easily replicable with LLMs, because LLMs are really trained more on sort of big things and not efficiency. There's languages as well. LLMs aren't good at writing an assembly or C today. So a lot of times companies come to us and want to license our technology, and that's the business that we're in. And it saves them time to market. You know, if they were to hire the right people and, you know, download some open source and then find the data for the open source and then tune it all, you know, they could spend years and not be as good as us. And when they do an analysis, uh, you know, they, a lot of them measure accuracy with their own data and they realize, wow, Sensory is really good and it's going to take us a few years to get there. So let's just go to market with Sensory and then work on it on the side and then replace them in a couple years. So it's always a challenge for us to stay ahead and make it so that two years down the line, when they do have their own technology, we stay ahead of them.

Speaker A: That makes sense. Yes. Yeah, you're describing a reality that is, again, we see a lot in the tech world, but I do love your optimism there. So, yeah. Thank you so much, Todd, for joining us to the podcast. If anybody wants to connect with you in the near future. Um, or just learn more about Sensory. What would be the best way to do that?

Speaker B: Sure. Our website is sensory.com, so we we own the sensory.com domain. People are welcome to go there. I'm on LinkedIn. Moser M O Z E R I think there's not a ton of Mosers. I'm Todd Moser on LinkedIn.

Speaker A: Great. Thank you again, Todd. And for everybody else who listened to this episode, if you enjoyed it, make sure you giving us a follow, give us a review or share this episode with someone else who would benefit from hearing it. Thanks again, Todd. And thanks everybody else for, uh, listening and see you next time.

Speaker B: Thanks. Thank you. Raimi.