EP 226 - Neuromorphic for LLMs on the Edge

Speaker A: He simply wanted to try something different. He said, I want to do something that's very efficient. And of course the most efficient computational device known to man is our human brain. And so the term in our industry is called neuromorphic. And so Peter went out to set a design in his head about how to do AI model acceleration in a neuromorphic fashion, which is the most efficient way. Anil Mankar it was a long term, three decade silicon venture and building dozens of chips. They got together together and henceforth the brand name Akita, which is the brainchild of the two of them together for our neuromorphic technology. So they always had a vision, let's do it different, let's do it better and more power efficient.

Speaker B: Welcome back to the Industrial IoT Spotlight podcast. I'm your host Eric Walenza, CEO of AGP and our guest today is Sean Hare and Jonathan Thompson, CEO and Chief Executive Development Officer of BrainChip. BrainChip is a technology company that is producing a groundbreaking neuromorphic processor that brings artificial intelligence to the edge. In this talk we explored how neuromorphic chips reduce power requirements in order to enable more sophisticated models for edge AI, including LLMs on the edge. We looked at topics such as running a compact 1 billion parameter LLM entirely on device, how event based processing lets a sensor ignore 99% of useless data and use cases like seizure detection eyewear with all day battery life in a tiny form factor. If you find these conversations valuable, please leave us a ah, comment and a five star review. And if you'd like to share your company's story or recommend a speaker, you can email us@teamsiagrowthpartners.com finally, if you'd like to discuss technology, innovation and growth challenges in Asia, please email me directly at eric walenza@asiagrowthpartners.com thank you. Welcome to the industrial IoT spotlight, your number one spot for insight from industrial IoT thought leaders who are transforming businesses today with your host, Eric Walenza. Uh, Sean John, thanks for joining us on the podcast today.

Speaker A: Hey, thanks for having us Eric. Look forward to the conversation.

Speaker C: Yeah, it's a pleasure to be here.

Speaker B: Yeah, and I'm really looking forward to this one. I think this is one of the most critical technology nodes that's uh, being developed right now. This on device edge AI, we really are seeing quite a transformation in AI more generally. But I think on edge device in particular is a very interesting area for the industrial space where I am where a lot of the cloud solutions aren't really suitable for the use case. So really looking forward to getting into this market and technology with you. Sean, before we get deep into the company, we'd love to understand where you're coming from. So I understand from our conversation earlier you were not one of the founders, but you were brought in, uh, as a CEO in 2021. What's the backstory for the company? And then for yourself personally? Why did you decide that this is the, uh, challenge that you wanted to take on at this particular point in time?

Speaker A: Sure. Brainchip has an amazing story. It has two founders, Peter Vandermeet and Neil Mancar, both of which have, uh, retired from the company at this point. Although Peter is on our board of directors still and he's on our scientific advisory board still, which is an industry group of people who advise some technology direction. But Peter particularly had a vision about what to do around computing. He looked and wanted to challenge the traditional ways of computing to be done, which is this very inefficient computation around AI using classic matrix multipliers, which is the traditional DLA model and things like that that the whole world uses. He simply wanted to try something different. He said, I want to do something that's very efficient. And of course the most efficient computational device known to man is our human brain. And so the term in our industry is called neuromorphic. And so Peter went out to set a design in his head about how to do AI model acceleration in a neuromorphic fashion, which is the most efficient way. Anil Mankar. It was a long term, three decade silicon veteran building dozens of chips. They got together and henceforth the brand name Akita, which is the brainchild of the two of them together for our neuromorphic technology. So they always had a vision, let's do it different, let's do it better and more power efficiency. I mean, you cannot read the paper today, more than one minute and see the challenges in the world about power consumption. AI, they were one of the early thinkers and we continue to lead that charge about that you can do it differently and much more power efficient. And we'll have a lot of conversation here about why the edge makes sense as well. But that's really what drove the company. You asked me a little bit myself, you know, I'm a long term tech person. I've been in this industry for many, many decades, work some of the biggest companies in the world and some of the smallest startups the world. And you asked me why did I want to come take this challenge on when I was given this opportunity. By the board of directors. I did some very important steps. You know, I was living in Silicon Valley at the time and had access to some of the brightest minds in the industry. And I asked them to take a good hard look at the technology, and they all gave me a resounding thumbs up, incredibly forward thinking, incredibly sound breakthrough, et cetera. I then talked to the team, and I talked to the team here, and I saw we have some of the very best scientists and engineers in the world. So I was very, uh, enthused by that. And then I had the opportunity to meet some of the prospects and customers and see the passion in their eyes to deploy our technology. It was just something, quite frankly, I could not resist at that point.

Speaker B: Yeah, great. And really a timely period to be getting into this technology, given the trajectory of the, um, fundamental technology base. I think as we go through this podcast, we're going to have to define a few terms. So you mentioned neomorphic computing. This will certainly come up. So can you help us understand more what is a neomorphic chip? How does it differ from a more traditional chip? You've already mentioned that a little bit, but I think it'd be important to understand in more detail. And then what's the difference between TOPS and sops in terms of how do we measure performance here?

Speaker A: I think this would be a good time to let John talk a little bit. John, why don't you take a definition first and I'll add to it if I disagree.

Speaker C: Yeah, sure. So neuromorphic computing is very straightforwardly trying to build chips that sense the world and compute like the human brain. And the argument goes that the human brain is still five orders of magnitude more efficient than the computer chips we have right now. So if we can understand how the brain works and instantiate that in silicon or in other compute technologies, we'll get some of that efficiency and some of that marvelous creativity going on. And then the question about TOPS and sops. So tops would be teraops per second, describing the rate at which a regular chip can perform computations. SOPS is not a widely used term, but it could be spikes per second. It's more a sense of how quickly a, uh, neuromorphic chip is having to do to get the job done. So they aren't really equivalent terms. Most regular silicon people would say a chip that does a high number of tops is good, it's doing a lot of computer. And most neuromorphic folks would say chip that's doing high SOPS is bad, it's using a lot of spikes to get the job done. So there is a little difference there in interpretation. And generally speaking we find both of those terms quite unhelpful because firstly, they can be gamed and secondly they don't um, connect to how good a chip is as well as you would expect. So we prefer to just say how much energy are you using to do an inference? How many joules per inference? That's probably the most ungamable, closest to real common sense assessment of how well it works.

Speaker A: I hope that makes sense, Derek. In the end, people are just looking for efficiency. Right. And so when Jonathan says joules per inference and things like that, it's really how little power can you do to do the inference in your model? And that's where we focus. Exceptionally efficient.

Speaker B: Yeah, it makes sense. I was a little bit struck by a comment I read maybe, uh, two months or so ago. I'm not a deep expert in this industry, but that if we look forward 10, 15 years and we look at competitiveness between countries, it really all comes down to power. Because everything else, they'll hit these bottlenecks and power is going to be the fundamental bottleneck that at least in the west, uh, we will have trouble getting around. I think in China, where I'm sitting in, they're able to set up power infrastructure pretty well. And so efficiency is less important in the west tends to be quite difficult. Right. So we really have, you know, we have strong technology in many layers of the stack. You know, kind of power bottleneck is going to be really significant.

Speaker A: But let me, if I could just make a comment to that, it's not just absolute power. I agree with all your comments. East versus west, the difference in there. But in the particular use cases themselves, even if you had unlimited power, it doesn't mean you have unlimited power for the device and the application that you're doing. Right. And so that will always matter. So if you're doing something that is in a battery or in the field or something like that, that efficiency matters for that device and so that metric will never go away. I understand the comment where you're saying absolute power. Yeah, that makes sense for big models. But when you talk about edge, absolutely, efficiency matters. And we'll get into this, I'm sure as we go through this. That's the beauty of AI. In the end it all starts like a lot of compute models, very and very data center centric, like all compute models, that it becomes very apparent to people that at some point you want to work, move some of the workload out of that data Center. I always like to say the right tool for the right task. You don't have one size fits all for any other compute in the industry and I should do for AI. So their efficiency will always matter on the edge.

Speaker B: Yeah, great point. Well maybe this is a good time to get into the use cases. So uh, we'll come back to the technology and get more into detail. But I think it'll be helpful to understand where is this technology applicable. So on your website you prioritize or highlight, um, hearseasense. So those are the three kind of clusters of use cases and there's one that I found kind of quite interesting which was a seizure prediction smart glass that you've co developed with a partner. And so when you talk about some of these things that you just mentioned about power efficiency and the importance of uh, all day battery life for high performance solutions using a tiny form factor, you know, I think this could be a great example. But can you walk us through a bit at a higher level? What are the clusters of use cases and maybe we can then dive into this seizure prediction smart glass and understand why is your technology important for uh, specific applications such as this?

Speaker A: Sure. I mean, you know one of the pleasures of my job is I get ah, to talk to many different prospects or customers around the world on a daily basis. And I'm constantly amazed with the use cases that are put before us is saying hey, we're going to do this and whatever that this happens to be. And so the way what we do here is we supply enabling technology for companies that are building devices to do something. Right. And so if you're going to create, whether it's a seizure detection glass or a piece of defense industry gear that's going to be out in the field or something like that, you have a vision saying uh, this is what I want to accomplish with that device. In order to do that I need some AI models running locally and I need to run them really well and that's when they engage with us. So you know, around the use cases that we constantly get, we see a lot of trend on is these cases where mobility matters, latency matters, battery life matters. And of course all that is the things like we just said, medical devices for sure, defense industry for sure and industrial IoT for sure. And even quite frankly in automobiles right now automobiles are nothing but a group of computers and sensors rolling around right now. And since the whole world is going to electric cars, the more efficient these devices are in the car, the better it's going to be. So those kind of industries are all interested in this technology.

Speaker B: When you look at what are the critical decision factors. So uh, when you're, you know, you're talking to a potential customer, they're evaluating, they're probably trying to educate themselves also about how should I be considering my tech stack? Right. I think there's a lot of companies that the industry is evolving in many areas quickly. Right. So they're probably not also crystal clear on how they should be prioritizing the different specifications. But as you're discussing through what tend to be the critical one or two or three specifications that would direct them toward, I mean you've already mentioned this a bit latency and so forth, uh, you know, power. But you know, help us understand a bit more. How does that factor into the business case for your customers? How does that factor into the functionality or the type of capability that they're able to bring to the end user? Yep.

Speaker A: When you're talking about running models, you want to, you're going to run them on a piece of silicon. And so every customer in the world is trying to get the most performance out of the smallest piece of silicon as possible at the lowest power. The industry calls that PPA Power Performance area. And so when we talk to prospects and customers, they're always saying we need to optimize that specifically for our use cases. You don't want to overbuild it because you're going to use too much power, take too much area, you don't want to undersize it. So you really want to get it absolutely correct around the PPA associated with it. That's one of the first criteria. And so it comes back to that efficiency is what can you do in a very small piece of silicon. The next thing is ease of adoption for us. Even though we've got groundbreaking performance, Most of the industry engineers have been trained on a couple of industry frameworks for creating models. If you create those models on the industry standard, um, framework, you deploy our software stack, you can then put them onto our platform. Very easy. So world class ppa, ease of adoption, those are the biggest criteria. And then I would also say for what customers, when I talk to them, one of the things they like about Brainchip quite a bit is this is all we do. There are other companies that perhaps maybe do something like this and then have a bunch of other offerings, but this is all we do. We're highly, highly focused on this m mission of bringing the very best edge technologies to us and doing that. So you have the entire company focused on it. And then one more thing, I'll pause and I'll ask Jonathan if he wants to throw one more on. There is, we've also made some noise as of late is we have a uh, proprietary set of models that are called state space models. Our proprietary name is called tens. So now we have the most efficient silicon. We're also working in exceptionally efficient models as well on that silicon. For those customers who are on tech, a little bit of both. Jonathan, anything you want to add to that?

Speaker C: Yeah, sure. Obviously we're a neuromorphic chip company, but the customers by and large don't much care what inspired the technology. They care only about does it give them a significant advantage in their system. And one of the things we have is that our neural compute engine can run without any intervention at all from a host processor. So almost all these systems consist of a host processor, uh, that's doing the high level stuff and a uh, compute engine that does computation efficiently. But in our case they can put the host processor to sleep and our compute uh, engine can just run in the background. And that's kind of our secret for as you mentioned, these situations with tiny form factor and long battery life, the host is asleep and the Akita chip or IP is just running in the background waiting for the particular event that will cause it to wake up the larger system. And so we're, we've got, as Sean was alluding to, we're now looking at a couple of extremely small form factor systems, uh, particularly the one we call Pico, which will be this kind of tiny little piece of silicon that can run in the background and wake up the big system appropriately when that is required. And so I think that's a tremendous competitive advantage. And as I say, customers don't much care how it does it. They just know that they're getting a tremendous performance advantage.

Speaker B: I completely agree, customers care about the performance advantage. But I would like to understand a little bit more how does it do this? So I mean, I guess there's other systems using more traditional hardware that could have one chipset that's running in the background and just doing a kind of preliminary process on data and then waking up another chip. So how do you differentiate your capabilities from these or what makes you uniquely able to do this in a, um, more efficient way?

Speaker A: Yeah, so I'll give you the high level one because I think it's always good to start high. I don't know if you've ever heard the term event based. Basically we take full advantage of sparsity, meaning unless there's a change, we don't do any compute on it whatsoever. So when you think about the edge environment, there's a lot of everything on the edge is typically streaming because that's the environment that the edge happened. But if there's no change, we don't do any compute. So that allows it to be incredibly efficient. That's probably the first and foremost element associated with that, which is take full advantage of sparsity. Now we talk a little bit later on about the models, some of the models we're developing now which give you the same level of accuracy, we're developing models that are third the size of other competitor models. So if you've got a really efficient piece of silicon that takes full advantage of sparsity and doesn't compute when there's no activity, and then you put on a very efficient model, you got something that's much, much, uh, more efficient than anything in the industry. Now, the contrast to more traditional ways of these things, in a classic matrix multiplication or multipliers kind of, uh, accelerator, they're always going to compute whether something's happened or not. So you can see the inefficiency where we ignore it. They're computing. Jonathan, anything you want to add to that?

Speaker C: Uh, I think you covered it pretty much all. Just to say, I mean, we sent only the most efficient computation is the one that you didn't do because you didn't need to do that computation. So if you can actually avoid doing computation at all, it's tremendously efficient. And our systems are just looking for change. If something changed, then they'll compute. If nothing changed, they won't compute.

Speaker A: And then I'll throw a couple more, Eric, just a little refinements on there. You want to minimize data movement and things. And we've got memory very close to the compute elements, so you're minimizing the data movement. Again, all these efficiencies add up over time.

Speaker B: And you mentioned one other key aspect of your proposition, which is, uh, ease of adoption. So I understand that that's critical. Can you help us understand a bit more? How do you, uh, provide that ease of adoption? And how does working on your platform differ from working on a more traditional AI accelerator or a TOPS oriented platform?

Speaker A: Yeah, again I think this would be a good one for Jonathan and I to tag team. But just what I said earlier, most of the industry creates models on a couple of traditional frameworks. We support them. And then you just basically utilize our platform called MetaTF, which is our software stack within Jeff, which actually maps it to the hardware. It sounds pretty simple, but it's really just that simple. And I like to give a lot of examples that it can do. We bring, um, students in here all the time. They learn it, they do models just like that. We have, I think, about a dozen interns here this summer. They learn meta TF and importing models and no big problems. So it's just really that simple. If, you know, industry framework, we train a little on a meditf, and then you map it to the hardware, it's just that simple. So you want to keep the creation of the models familiar for the hundreds of thousands or million machine learning engineers in the world, but make it simple to put on. That's at the very basic core of it.

Speaker B: Okay, got it, got it. Okay. So it's really just around adhering to industry standards and avoiding adding complexity.

Speaker A: You know, it's interesting, Eric. It's because it's defined. Actually, you know, I did a podcast, we do our own podcast here, and I did one about a year or so ago with a very famous business author, uh, a friend of mine, Geoffrey Moore. And his most famous book was Crossing the Chasm. Right. In that book he talks about technology, which is undisputably better. But sometimes it doesn't cross the chasm to mass adoption because people don't want to have change or it's too hard to change and adopt. So one of the key principles we have here at BrainChip is we want to get that breakthrough performance that we've been stressing the first part of this podcast. But we want to make it incredibly easy and friendly for people to move their models onto our platform.

Speaker B: Makes complete sense. Makes complete sense. Um, one other aspect of this that I think is quite important for our listeners to understand is to what extent the work that you're doing is pure on edge. So let's say it's a pacemaker, it's making a decision and it's acting and it's 100%, um, confined within that device versus on edge plus cloud, meaning that it's doing some kind of pre processing and maybe it's then communicating to a cloud system. What have you seen in terms of the mix of the use cases that you're working on on a pure on edge versus a hybrid model?

Speaker A: Yeah, I mean, most people approach us for the pure on edge elements because that's where we excel, but there's absolutely no limitations in that technology that you could not do. Exactly what you're describing, what I said earlier, do the right, compute at the right place. And if you got to do more later, then you do it more later up in the cloud. But no, right now the vast majority of the conversations are driven by on edge computing.

Speaker B: Got you. So if we drill into on edge, can you help us understand where is the cutting edge or where are we pushing the boundaries in terms of capability on edge? Right, because of course there's compute kind of bottlenecks because of the space available, there's power bottlenecks. And so what are we able to do on the edge in the more ambitious use cases that your customers are bringing to market maybe today? And then if you can help us understand a bit more of the trajectory, if we look forward 12 months, 36 months, what are you anticipating that will be possible on edge that might not be possible today?

Speaker A: We get about three hours to talk about that. I would start first by saying this is an industry of rapid change. It does not stand still. Now I've had the honor, uh, of being the CEO of BrainChip now for about three and a half years. And the model types that we support today were just dreams when I, when I took this job. And now these model types are mainstream and that pace of innovation is not slowing. It's coming quicker and quicker. And so what it requires is companies like us that offer acceleration technology is to ensure that we keep developing our platforms that can keep supporting all the model types of the future. And so those capabilities are limitless, depending on how people can dream, what kind of models and what capabilities they can bring out. You know, I'm glad Jonathan's here because we just did our shareholder meeting last month and we, you know, Jonathan showed our roadmap and in there we've got some very interesting kind of future stuff that when I say future, it's not real long term future, including LLMs on the edge. And I believe you were going to ask me some questions about that a little bit later, but it'd be a good time to introduce that right now. Just the thought of those capabilities maybe 18 months ago was really, uh, conceptual. Now it's very, very practical. We've got very strong customer interest. Our deployment, that work we, that we have is unique in this industry and we are beating the industry on all those PPA kind of things I talked about earlier. By a factor, we're not talking 10%, we're talking X number of times better. And so the ability to deploy the functionality of an LLM on an edge device, I think that speaks a lot in a few short years. Jonathan, anything you want to add to

Speaker C: that, yeah, sure, there's a kind of threshold with LLMs, for example, that if you have less than a billion parameters, you just don't get a really human kind of response. It's not grammatical, it's not logical, it's words, but it's word salad. And above a billion, suddenly there's almost like a phase change and you get a human quality response. And so our ability to put a model above that billion parameter threshold on the edge with no web connection or cloud connection, uh, is transformative. And I think that's the most exciting thing that we see coming along in the next two or three years. And again, to Sean's point, I mean, you know, these models got bigger at a rate of four or five times per year. So, uh, you know, that's 125 times in three years if you compound it, so to speak. And we really have seen that growth and been able to keep pace with it and are going to be able to deliver at that scale, which is great.

Speaker A: I would say one thing, Eric, if I could. This company, the vast majority, I don't have the percentage here in front of me, but the vast majority of this company is nothing but scientists and engineers. We have, uh, world class scientists that are doing nothing about keeping abreast and leading model advancements and world class engineers building the best silicon and software stack. That's all we do. We're thinking about this night and day about how to get these models to run exceptionally well on the edge.

Speaker B: So Jonathan, it sounds like LLM on the Edge is not quite here yet, but you're expecting it within one or two years to be, uh, capable. Can you explain?

Speaker C: Yeah, I would say less than that. Yeah, we've demonstrated it and we expect to have a product by the end of the year.

Speaker B: And can you help us understand what would that first product look like in terms of battery life, in terms of constraints? Is there any more detail about that first generation of LLM on the edge capabilities that you can describe?

Speaker C: I think I'm going to be cagey about that because the sales guys will kill me if I oversell or overexpose anything. I might give Sean that one to tackle. He's probably got a better picture of that. I can give technical detail.

Speaker A: No, I think the right answer to that, Eric, is what I said earlier, which is we know for a fact that we will be able to get better performance to find, uh, whatever performance level at a token rate or something like that, at a power element that will be many times lower than the industry right now. If you think about that, that's a big bold statement. Where most technology companies make a claim of being 10 or 15% better, I'm making a claim that we'll be many times better.

Speaker B: So right now there's a company for example, that uh, we're working with that does a little snap in clip. You go into a meeting and you say, hey, I'm going to be recording this meeting. Or there's other ones that will kind of do a direct translation. They tend to be a bit, a bigger device, like a pen shaped device that will do a direct translation. So are those, uh, I'm actually not familiar with the tech stack those. Are those all connected to the Internet or are they able to do. I guess that the PIN doesn't actually do any processing, it's just recording and later it'll translate. The pen actually does the processing in real time so it can translate. Is that doing that via cloud computing to your understanding, or is it just a larger form factor that has a more like a kind of sizable compute on the device and therefore is able to do these types of jobs today?

Speaker A: I think it's a little bit of both, but I think the predominant model is a cloud model and that's why the latency is usually pretty bad on these kind of things to do it. But there are some devices that are coming to market or out in market right now that can do it local. Not quite as well yet, but they will get there.

Speaker C: If you're still listening, then you must take technology seriously. So I'd like to introduce you to our case study database. Think of it, uh, as a roadmap that can help you understand which use cases and technologies are being deployed in the world today. We have cataloged more than 9,000 case studies and are adding 500 per month with detailed tagging by industry and function. Our goal is to help you make

Speaker A: better investment decisions that are backed by data. Check it out at the link below and please share your thoughts.

Speaker C: We'd love to hear how we can improve it. Thank you. And back to the show.

Speaker B: There's a topic that's come up, um, in a couple of conversations I've had recently and I don't know if this is pertinent to your tech stack or not, but it's the content, the concept of kind of a mesh network of edge devices that are doing AI in consortium. So if you're talking about drones, for example, in a military use case, yes, each device kind of has its own computer, but they're also coordinating together. Is that something that is kind of important to the use cases that you're working on, or are you more typically working on, uh, use cases where the device is kind of, uh, acting independently? And I'm asking that because hypothetically then if you have 100 drones communicating together, you would have 100x. The compute powder potentially could do more sophisticated computations than you could with one individual, uh, device.

Speaker C: We're not actively working on any projects like that. We do keep an eye on that space. Uh, we are very interested specifically in the drone space. And my observation of that field is that it's not nearly as mature as it's sold to be. And one of the specific problems is if you have a single device and it's computing to the fullest extent of its ability, and then you network it to a bunch of other similar devices, well, none of them actually has any capacity left over. You can swap, uh, information, but you have to have spare capacity to make use of that information. And that tends to get forgotten in the sort of rosy picture of networked intelligence. Seeing and bidding on some opportunities where that data is getting fed back to a central system, central AI system somewhere. And I see that as having immediate value and doable in the short term. But the kind of distributed swarm intelligence is a much harder problem than it's. It's generally paid.

Speaker A: And I would probably just amplify what Jonathan said on the front end. We are, uh, in conversations with, with several drone companies because if you think about what we talked about, the value proposition, exceptionally high performance and a small piece of silicon, long battery life, it's ideal for those kind of devices, right, that are trying to have extended life out in the sky.

Speaker B: Gotcha. Well guys, I'm obviously not an expert in, uh, AI on the edge, and so I'm sure there's a lot of things that I'm missing in this conversation. What are we missing here that's really important for folks to understand, whether it's on the technology side or on understanding which applications are feasible and how to optimize the outcomes for the solutions that your customers are building.

Speaker A: I think it's an awareness thing. I mentioned earlier, I've been in the tech industry for several decades and what I said earlier is right is the right tool for the right task. Again, AI itself, if you think current bloom of AI in the last five, six years and all the energy and excitement around that, the edge market is still a little bit behind it, but it's growing very quickly. All the analyst results or reports are talking about the edge market being the fast grower. In fact, there's a guy who read an analyst report the other day, said this simple last 10 years was all about big data center and training. The next 10 years is about edge and inference. Right. So it's just the right tool at the right time right now, as people start to wake up and think of these kind of things that they can possibly do that they never could do before. Now, you talked about one with the seizure, uh, prediction glasses. Well, who would have thought of that? You can't do that at a data center. We have a partner right now that is flying around on a drone scanning a beach, looking for distressed people in the water. Couldn't do that. So these use cases, she's coming up and more and more. So I think we're just really at the beginning of this. And so we're going to see a lot of growth in this market. Again, every analyst in financial or technology saying, edge technology, edge AI is the future. Right now.

Speaker B: Sean, when you're having sales conversations, where does this typically start in the organization? Is it to the point where they've already, you know, engineering is kind of specked in that we need certain edge AI capabilities. And then procurement contacts you and says, we're doing a tender and we evaluated the top five vendors and we'd like to, uh, invite you, or is it more kind of having early discussions with the engineers about, hey, here's, here's what's possible today. Let's educate you and kind of work together on the product development. How, you know, how mature are the organizations that you're working with in terms of understanding the landscape, um, of what's possible and who the vendors are and how are you approaching those conversations?

Speaker A: Sure. When you think about if somebody's going to buy AI acceleration technologies, they're doing it to put into their core product, which is their revenue stream. It's a very, very strategic decision. To answer your question directly, most of the conversations originate out of the product management function, where they have this vision for this thing they want to build and they recognize they need some capabilities. They're trying to figure out who can do that. It's not at that you, uh, said the procurement level. It's usually the product management, which usually has some kind of function from the CTO office or somebody technical to say, I picture my product doing this Mr. Or Mrs. CTO office, help me find the right technology to do this with the characteristics. So it's usually we call them a technology scout and product management that we work with. And usually when they Come to us and say, hey, we've got this thing, we want to do this. Will it run this kind of model at this performance? And we do. And we have a lot of simple ways to try. I mentioned MetaTF. They can simulate it on MetaTF. We can give them, um, a little development kit with one of our chips on there. They can do it themselves, they can send it to us and we verify it for them and show the same and say, yeah, here's your results. Then you go into the business conversation.

Speaker B: Gotcha. Right now, your business model is it tend to be. So you have a development kit, they can, uh, do a quick POC and make sure that it functions. And then are you typically selling this more from a project perspective of saying we're going to work with you together, some kind of fee plus cost per device, or is it typically just a pure, you know, once we figure out that this works for you, you're paying per chip. I know you also have the software stack. What does that typically look like? Is it just kind of a per chip or is it like, uh, integration plus chip plus software, uh, license?

Speaker A: Yeah, well, a couple of things. Our primary business model is we're an IP provider. So if we help people who want to build their custom chips, we solve an IP block called the MPU neural processing unit to do that. And so our primary business model is licensing of that IP for people that can design a custom chip in there. And so like all IP licensing, there's a license fee associated with it and then there's a royalty associated when those chips come off the production line. We get a percentage of sales or ASP for that for every chip that goes forward. Uh, we do typically charge some NRE to help them with the project side like that, um, depending on the customer. Some have outstanding machine learning capabilities and they're totally fine themselves. Some ask us to help them with their models and we can charge them for that. But so, but that's our primary business model is around, you know, ip, you know, we sell on the license, get royalty, but we also help them with their models if they need to as well, because in the end we're experts in all of that. So we can help them in that case.

Speaker B: Yeah, got it. And for your, the customers you're working with right now, does it tend to be more, you know, kind of medium sized, uh, kind of OEMs that are building some new, you know, build. They're building drones, they're building a smart glass or some kind of, I would call it kind of a New, a new category product or does it tend to be, you know, kind of the very traditional companies. We're doing construction equipment, we have sensors now we'd like to do some on sensor. Who are the kind of the first adopter, uh, wave. Does it tend to be more of the um, mid sized new category type companies, uh, or the traditional products?

Speaker A: I don't like to give kind of vague answers, but I will this way. It's a little bit of both, but if I had to pick one, the medium sized companies are more innovative but typically large companies are very interested too because the larger companies know that their business models are under threat if they don't step up and keep making competitive products. Meaning their products, they have to have more and more functionality. So we see it from both sides, very large companies and more aggressive mid sized companies that are trying to penetrate market with innovative products. Those would be the classification of that. And also I think I want to draw another dimension since we're talking about business models. Some companies are building products so they're going to build a custom piece of silicon, go in that product and they're going to be captive. And there are also companies that say I want to build a custom piece of silicon to sell to other companies and they'll make that so they can sell and go into multiple companies. So the way to look at it is almost a 4x4 matrix or a 2 by 2 larger companies, innovative mid sized companies and those that are building for their own captive use and those that are building chips for the market that sell into the market.

Speaker B: Yeah, got it. Yeah, we actually invested in an event based platform, uh, probably now, six or seven years ago. And so I guess they would be the type of company that they're building a platform, but potentially they could also be designing a chipset to use with their platform on your IP and then selling that through to their customer base. Is that kind of uh, additional category of uh, customers there?

Speaker A: I'll take that sales lead, Eric, after the call.

Speaker B: No, but the reason I'm asking, absolutely, Sean, is that we work quite a bit with the larger companies and we really do see that there's a significant challenge at a lot of these. You know, they've had a lot of success in the past. They have certain business models that have worked well and so it's just often much more difficult for them to figure out how are we going to modify our business model to make use of this new capability. Whereas these kind of medium sized, they're a bit more like digital native companies. The business Model of using data and, you know, providing services on top of hardware was baked in from day one. Right. So it's much more intuitive for them to figure out how to use, use, uh, these technologies as they mature. Whereas the more traditional ones, it's just harder to shift. Uh, once you've had a model that's been successful for 30, 40, 50 years, kind of selling mechanical engineering in a lot of cases, or electrical engineering and now selling data enabled services on top of that. But of course that's such a huge part of the economy that that's really critical not just for your business but also more broadly for adoption of the technology that these companies figure out how to make effective use of this technology as it matures.

Speaker A: Absolutely. Well said. But I would also say if you asked 20 years ago, would you ever think, you see the day of these old traditional car companies, that the vast majority of their product line would be electric? People have to change. And the whole AI revolution, I personally believe, and many analysts believe, is going to be a greater productivity revolution than anything we've ever seen. Whether that's the information age, the industrial age, agricultural age, the productivity enhancements to our world are just going to be unbelievable and every company will be affected. And those that don't decide to modernize their product lines, I don't think we'll survive.

Speaker B: Yeah, agreed. Well, in automotive we're already kind of seeing that. Right. So um, you either adapt or you're just, it'll be a slow death. Right. These are behemoths, but you adapt or you're not going to be around in 50 years. Sean, last question. So if some of our listeners are interested in working with you or understanding more about your technology, what's the right way for them to reach out to the team?

Speaker A: Absolutely. I would love to Talk. We're@brainchip.com and there's a way, just go to uh, I think it's called salesrainship.com. that's one. Of course they want to reach out directly to me. I'd be glad to um, do that or direct anybody in our sales organization.

Speaker B: Awesome. Well Sean, thanks for taking the time today. Really appreciate it.

Speaker A: Likewise. It was really fun conversation. Eric,

Speaker B: Thanks for tuning in to another edition of the industrial IoT spotlight. Don't forget to follow us on Twitter, OT1HQ and to check out our database of case studies on iot1.com. If you have unique insight or a project deployment story to share, we'd love to feature you on a future edition. Write us at Eric Walenza@iot1.com.