S:8 E:2: Neuroplastic AI models

In this episode of IoT Unplugged, we find out about neuroplastic AI and how it's being used in industrial IoT devices. Today, I'm speaking with Luffy co-founder Matthew Carr, an AI entrepreneur and former nuclear physicist who has developed an AI model that can be trained in simulation without large data sets and claims to be up to 400 times more efficient than traditional deep learning. Today I've got Matthew Carr from Luffy with me. Pleasure to have you here on the show Matthew.

Before we start talking about neoplastic AI, I wonder if you can just give me a brief introduction telling me who you are and what you do. First of all a bit about your background please, I think it's quite unusual. Sure, hi Lucy and thanks for having me on the podcast. I've always had a passion for industrial technologies and also sort of control and power generation and also always been passionate about the energy transition and trying to work closely with technologies that might be able to help with sort of climate change and sort of the energy transition.

So I sort of started my studies in engineering, working on sort of solar cells and wind turbines and renewable energy. I then moved over into physics so I could kind of go deeper into the technologies and understand the algorithms that sort of control them. And that eventually led me into fusion and sort of looking at fusion as a new energy source. I then moved to the UK, about 10 years ago now for my post-op working at Jeff's, the world's largest fusion reactor, which is an amazingly sort of complex machine where you have, you know, 10,000 sensors and a lot of different control and physics going on.

Temperatures of up to 100 million degrees and lots of sort of advanced research. And as part of that, I sort of met my co-founder. We were working together on sort of data science and optimization, and he'd been doing his PhD research in advanced AI techniques, such as new revolution. And so sort of what led us to Luffy is actually the intersection of those sort of interests.

So my background around energy transition, manufacturing, advanced technologies, his background around these sort of advanced AI and data science techniques. And that's quite a journey from being a nuclear physicist to being an AI founder for an AI company. So tell us about Luffy. Why Luffy?

What's the meaning behind the name? Sure, yeah. So Luffy comes about from our sort of noticing a gap in the AI landscape. We could really just see that so many of today's AI technologies were really focused on sort of very large problems where you've got a lot of data that's sort of rich and abundant.

But as we know, when you start going more into the IoT world and sort of edge use cases and edge devices, that gets increasingly challenging because you tend to struggle with the data and compute requirements of those occupations. And so we were really sort of seeking technologies that could help us with that gap. And so we all sort of see AI and autonomy as key technologies that will help us sort of reassure our lost, you know, sort of manufacturing industries, compete with low-wage economies, reach our decarbonization targets.

But despite all that potential, what we're mainly seeing in manufacturing is dashboards, you know, image-based fault detection, sort of rich sort of predictive maintenance. But where's the transformer moment for edge devices? They're going to have sort of rich AI control embedded in a pump or a fan or a motor. So that's sort of what got us really excited about exploring that space.

So yeah, so we were exploring that space as part of our research over a couple of years and could just sort of see that there was a need for this new type of architecture. And the potential is massive. You know, McKinsey is estimated there's around 100 billion in value to be unlocked if we could deploy AI into these sort of edge devices. The International Energy Agency estimates there's between two to six percent energy savings that could be achieved if you could embed AI into, you know, pumps, fans, HVAC, engineering systems.

You know, all of these things can seem very boring on the surface of it because there's so many of them, but something like half of the world's electrical energy ends up being consumed in these devices. So if we can make them smarter and have sort of AI at the edge, it's a very, very big win. And why Luffy? Why the name Luffy?

Luffy was actually originally a code name for us. So basically, while we were working at the Atomic Energy Authority, we were doing sort of a bunch of different research projects. And so we sort of needed a sort of code name for the project. And it was actually the name of a friend's guinea Pig.

So it's actually a kind of a funny link there. We always had planned to come up with a better name later that was sort of more relevant to IoT and sort of Edge AI. But actually, we sort of noticed that a lot of the best startup names actually don't mean anything. They're kind of friendly sounding.

There's no sort of attachment to it. And the name just kind of stuck. So that's where we are with that. That's a wonderful reason.

So can you explain a bit about neuroplastic AI and how it differs from other forms of AI, like chat GPT? Sure, yeah. So I think part of it is behind understanding the motivation of why you would need neuroplastic AI in the first place. And the big reason is that if you're looking at a lot of these IoT devices and edge applications, we see that the mainstream deep learning techniques, they just really struggle with the data and compute requirements for that.

The thing we always hear from people using these devices is it's very difficult to get the large data sets that you would typically require. It can be very expensive. There's a lot of data modes that sort of restrict them. And then also the fact that the computer environment is typically very small You know you looking at a microcontroller more very small you know sort of edge device And so you can have these large language models and embed them onto a small microcontroller.

So it leaves this gap for a new architecture. And so when we were looking around a new architecture, one of the ones that really stands out and comes from my co-founder's research is looking to biology for inspiration. And if we look at all biological systems, Our networks are not static. It's not like the weights and connections in our synapses are fixed.

Actually, they're plastic. They change and adapt over time. We can learn new things over our life, and all animals have this mechanism embedded into them. Sometimes I like to think about it by analogy.

If you've ever seen a deer being born, a deer can learn to walk within an hour. What that tells you is that the pattern for walking is already embedded in the deer's brain, but in that first hour of life it's learning things like how much its body weighs how long its arms are how strong its muscles are and so the idea is what if we could take that biological characteristic and embed it into a pump or a fan or a motor so that this this sort of edge device can now tune itself to its body it could learn okay this motor needs to be more energy efficient or this air conditioning system needs to adapt to how many people are in the room you know these kinds of concepts so we want to kind of take that biological idea and embed it in and that and that's basically what we've done.

We've taken some research from medical research and sort of bio-inspired AI research and taken some of those neuroplasticity rules and embedded them into the network itself. And that's what is really the secret source that gives these networks the ability to tune and adapt, you know, when embedded into an IoT device and edge hardware. That sounds amazing. So if you train them in simulation and then you find them in reality, like your deer being born, what does that look like in practice in an industrial setting?

How can you actually simulate that? Well, it actually makes it much easier for the manufacturers and partners we work with, because typically these simulation models already exist. You know, when they design the equipment or the plant in the first place, they actually already have them. It can be something like a digital twin, or it could even just be some equations they use for the design.

And when you're talking about some of the things like we're already working with, like industrial motors and pumps and fans. I mean, these have been around decades and all the models already exist. And that's much easier to say, okay, we'll talk about which type of model we're targeting versus say, oh, give me your data set from the last five years for a hundred of your plants. It becomes much harder for them typically to share data.

They're more protective about their data. It's their crown jewels. It's their secret sauce. But if you kind of shift to instead just say, hey, let's talk about models, let's talk about digital twins, and a lot of them are actually already available, it makes it very easy to sort of collaborate and start training.

So we always start working on one of these simulation-based models. We'll train the evolution process that's mimicking natural evolution, evolving neural networks in simulation. We run that algorithm millions and millions of times. And what will come out the other end is a very small, very compact network that has these neuroplasticity rules in it.

And you can then embed it in the hardware and it will tune and adapt to that hardware. Wow. And I suppose by having that on the edge, then, does that mean that it's got a lower carbon footprint? You don't have a massive, you know, huge data set in the gas guzzling data center, that sort of thing.

Is that how it works? Exactly. You end up creating savings in multiple ways. So first of all, you don't need to have sort of communication with the clouds.

You don't need to be sending that data back and forth. The other issue is that now you have some of the learning actually happening on the edge. So with a more conventional system, you'd be training on data on the cloud. And then periodically, you might have to do retraining when something in the plant changes, or you've got changes in the data sets underlying.

But if you have a neuroplastic or adaptive sort of neural network embedded in an edge device, it can self-tune to changes in that device. Now, obviously, it's within reason. It's not going to learn to read. It's not going to become sentient, you know, nothing like that.

But what it can do is it can learn, oh, okay, maybe this motor's got a bit more friction in it, or maybe there's going to be a change in the way I need to run this motor. And that's the kind of thing it can learn and adapt to. And so this saves energy on the sort of data communication. But there's another really nice side effect, and that's that the neuroplasticity itself just makes the network very small.

So a more conventional network, like you said, you'd have to have a large data set, it gets embedded in the network, and that makes it very inefficient to run. But if you have a neuroplastic network, it's kind of just learning how to tune itself. It's learning how to adapt itself to that problem. And so in our benchmarks, we see that that can be as much as 400 times more compute efficient.

Obviously, it changes a lot depending on what you're doing and how complex it is. But we typically say from 100 times to 400 times more compute efficient. And that is massive because that now means something that might have been a graphics card or might have been a bigger chip. It can now be running in a Raspberry Pi and fractions of a Raspberry Pi.

So we've done examples where they're running in a drone or a motor and it might be running on 1% of a single core of a Raspberry Pi. And that's where you're getting massive compute and energy efficiency savings. But when I speak to AI founders, a lot of the time they say how we need to have massive data set We need to have all of this so that you get all the possible computations If you a protein without cloud intervention I mean how can you have enough space on your tiny little edge device to do all of the gubbins you need for AI?

Well, that's, I think, just the magic of neuroplasticity. And it's, I guess, why our own brains evolved in that direction. The way I like to think of it is it's just a more efficient encoding. You know, if I have to put all of that data into that network on that edge device, it's almost like putting a database there.

I'm trying to put all of that data into the device and then it has to kind of look up its database every time for every case is quite large. Whereas with neuroplasticity, I'm more just trying to find the essence of the circuit, the essence of the feedback, the essence of how the signal processing works. And then neuroplasticity is just using its adaption rules to kind of change how strong those pathways are based on what's actually happening. And so it's kind of like I'm trying to find the essence of the feedback circuit rather than making a database.

And that's how you end up getting all those savings. And you're right. I think most people are more thinking in the cloud direction, but I think that makes sense because if you think about, you know, some of these big AI companies that are pushing it, you know, all the FANG companies, they're rich in data and they're rich in cloud resource. So that's why they tend to think that way.

But if you're talking to manufacturers on the ground or people with a lot of industrial edge equipment, you know, there's a lot of silos, there's a lot of firewalls, some of it's not even connected to the internet. It might be out in a remote location. So that's not going to work for them. They need something that can fit into the compute profile of what they already have.

And that means neuroplastic, compact, very efficient networks. So these things actually been used in practice, where have you actually done it? Yes, we've done it in a few applications. So we've done some early experiments in drones, where we managed to put neural networks into the flight control of a drone.

And as I said, that's where we ended up getting a network that was around 1% of a single core of a Raspberry Pi 4. So it was very compute efficient and very impressive for that demonstration. It was able to, you know, fly basically with changes in the mass of the drone and sort of learn its body. And we even managed to break some rotors and actually it can sort of self-adapt to that body configuration and keep flying.

Just like if you sort of had wounded your leg, you'd start limping, but still be able to walk. So we really managed to sort of demonstrate that capability. We also have a company where we work within injection molding, where they have these sort of big molding machines that sort of make different parts. So like, you know, plastic components and carbon fiber composite components that would be for, you know, automotive or aerospace.

And in that case, we managed to make an AI controller that could sort of improve the energy efficiency of that machine by almost 10%. And it also saved tuning, which means you sort of don't need to manually tune the machine as much anymore. So these are some of the near-term examples. And we're now working with some of the biggest companies on the planet in industrial motors for pumps, fans, logistics.

Fortunately, I can't name their names, sorry, but they are some very big companies I'm sure you've heard of. I'm working with them on embedded AI that can get embedded into their motors. So you could now have a plug and play motor that you don't need to configure anymore. You just plug it in and it self-tunes to the load characteristics, et cetera, it's running.

And this will create energy efficiency improvements and performance improvements for these applications. So what does it actually look like in reality? Is it like a little box that you plug into a motor? And what does it do when it's in the motor?

So it's actually just pure software. So actually we have a small bit of code. It's typically about 10 kilobytes. So absolutely tiny in memory footprint.

And then we embed it actually into the microcontroller of the motor. So in this case, you know, we can input it into that microcontroller and we can achieve very sort of tight compute time. So around a millisecond sort of inference. Also, we can put it into a device nearby.

So it could be a Raspberry Pi or a PLC. So there's sort of different options really depending on your setup. But I think the mission here is to kind of make very flexible neural networks that can be targeted at, you know, sort of specific applications en masse and then easily embedded to the hardware you already have. You know, and that will be the key to making this technology scalable and realizing that is sort of gains in energy efficiency and performance.

And is it compatible with everything that's out there already? Or if you're a manufacturer, would you need to invest in a lot of expensive software before you could use it? It's sort of not yet compatible, I'd say, with masses of equipment yet. We're all sort of on that journey.

So we're sort of working with sort of phased applications. So we've started with logistics applications, looking at, so for example, an Amazon warehouse or a mail sorting warehouse, that kind of application. So we've already got some networks that work there. We're starting to work on pumping applications for wastewater, you know, sort of water utilities and page vac systems.

So we're basically adding a library of these controllers. And what we hope to do is over the next year or two, we'll have this library that you can almost pick and choose from and sort of work with. And then we can assist companies with the integration. Okay.

And what are the biggest challenges you've had to face when you're trying to embed Luffy's AI into hardware? Well, the biggest challenge originally was just trying to find the right architecture because it's very, very difficult. If we trying to find a neural network that can actually fit inside 10 kilobytes and fit inside the existing microcontroller that was actually the biggest challenge So we managed to sort of solve that with the work we done And then the next big thing is really sort of meeting the needs for integration requirements as you sort of hinting at So being able to tell these manufacturers or these sort of edge users that okay, it's going to be, can be integrated in a way that's cyber secure.

You sort of don't need to sort of change your architecture. It just becomes an easy way to adopt this technology. And once they adopt it, I mean, what could the implications be? How much more energy efficient, how much smarter could your industrial tech actually be?

So I think it changes a lot from application to application. So as I mentioned in that injection moulding case, we saw it was 10% savings in electrical energy, which is definitely nothing to sneeze at. That's quite a significant improvement. For something like motors en masse, I think there'll be a range.

So it could be as small as 1%, but all the way up to 8%. And it probably depends on things like how well tuned the motors were in the first place. and also how much their loads are changing. Are you doing a lot of stopping and starting or is it just running kind of constant?

So the gains you see will change a bit depending on the application, but it'll be somewhere in that window of like 1% to 10% kind of energy savings. And if we want to sort of reach our sort of emissions targets, this is going to be one of the best ways to basically reach those targets is looking at all the hardware we have today and how can we sort of eke out more of performance. The other thing we see is that it has a big benefit on sort of labor requirements and commissioning and downtime.

Because if you now have something that can self-optimize, you don't need to be sending around engineers to kind of constantly sort of do maintenance with it. It can now be sort of self-automating. So we've been looking at some applications, like as I mentioned, like in wastewater and utilities or logistics, where you might have hundreds to thousands of these kinds of systems that are all sort of at the edge. And it becomes a lot of work, actually, to go around and optimize them all.

So there we just see that it will lock in a lot of benefits for sort of operators and sort of utility companies because you're sort of locking in that sort of benefit and performance improvement. So it means less downtime, less wear and tear, more uptime. So, you know, it's very valuable. Okay, but a saving of between 1% to 8% doesn't sound that much for, I mean, obviously it adds up, but it doesn't sound that significant if it's towards the 1%.

What sort of capital outlay would you need to get that 1% saving? Well, it's no capital outlay. It's completely software-based. So as I said, it'll really depend on the application.

It can be very, very significant. And if it's a pure sort of software upgrade that you can embed into a lot of equipment en masse, it will create these sort of very attractive operational savings. But how much would it cost for an industrial operator to invest in it to get that sort of saving? Well, I can't sort of disclose our pricing, sorry, on this call.

But I can give you a sense that it's a sort of a cheap software add-on that you might buy en masse. So if you're typically at the moment, we're targeting people who are doing large orders. So probably in the hundreds to thousands of these equipment at a time, working with some big industrial operators. But it'd be a sort of very minor sort of add-on to their sort of package that they buy.

Looking ahead then, how do you see neuroplastic AI shaping the next generation of IoT devices and smart industrial systems? Well, I think it just opens up new possibilities. So I've spoken to many people that were thinking if we're going to add AI capabilities to our edge devices, we're going to have to put bigger chips in. We're going to have to put in small, you know, neural processing unit.

We're going to have to put in a NVIDIA graphics card. We're going to have to put in some bigger compute. But kind of what we're doing is challenging that mindset. Because if with neuroplasticity you can get 100 to 400 times compute improvement, that now means you can fit into the compute you already have.

And so that means there'll be a sort of a lot of applications this opens up. I think the second thing is that so many of today's IoT devices were designed around the constraints of the past, actually. And so what will be really exciting for us is all the new devices that this might open up. So basically, we're seeing that coming wave of robotics, where we're seeing a lot of humanoid robotics.

So this will sort of open up that, you know, basically, if before you had to have a big graphics card in the back of the robot, but now you could have it on sort of more conventional compute. That would really open up new possibilities. Also, I think new hardware designs, even for industrial assets. It may be now that we can sort of add more sensing capability and this will create new opportunities in terms of manufacturing and robotics.

That sounds amazing. Is there anything else, you know, what could Luffy AI deliver? You know, what are the benefits that Luffy could deliver to businesses and society in the next five years? I think the big thing we're hoping to do right now is go after the industrial motor applications market because it consumes about 50% of the world's electrical energy.

And so if we can lock in some energy savings on that, that's potentially just massive, massive benefits. So that's sort of a big focus for us right now. But then after that, we hope to move into this coming wave of robotics applications and other types of process equipment that's coming. And really, we just want to enable sort of a revolution in these types of devices by having sort of very compute efficient AI technologies.

That means we don't always need to default to the cloud or bigger compute. Wow. Well, thank you very much for that. That sounds amazing.

And I wish you the best of luck. Thank you, Lucy.