AI Agents Are About to Move Off the Cloud. Here’s What Changes

Speaker A: AI is nothing but a piece of software, right? And all this software and all that, it has to follow the physical rules and there are a lot of hard rules and govern it. If you can break down the task to a, uh, manageable size of task and have very clear instructions, they can do it. So I think that's fundamentally the philosophy of today's hybrid AI concept. AI make mistakes and agents sometimes lie. Not, not sometimes lie all the time. And, you know, for example, the, the, the execution plan is laid out. You know, this, this agent has to call these tools, go search that information whatsoever and then report it. I'm done. Success. But it never caught that too. It never caught that too, you know, but it just lie back right after your face. Success.

Speaker B: Welcome, humans, to the Neuron. AI Explained. I'm Corey Knowles and I'm joined today by Grant Harvey. As always, Grant, how are you, man?

Speaker C: I'm doing good, Corey. How about you?

Speaker B: Oh, I'm doing good. Uh, excited to have this little chat today. Can you tell us a little about who and what we're going to discuss?

Speaker C: Yeah. So today we are talking about where AI actually lives, because most of us experience advanced AI through enormous cloud models. But a growing number of PCs and devices can run smaller models locally. And intel believes that the next phase will be hybrid. So an intelligent system that decides which tasks belong on your device and which require more powerful infrastructure. And for the record, I also agree with this. Today we are joined by Dr. Olena Zhu, who leads AI solutions and ecosystems for Intel's client computing group and has helped shape that strategy, including Intel's newly released superclaw.

Speaker B: Before we get started, please take just a quick second to like today's video and subscribe above so you never miss an interview or one of our live streams. And on that note, Dr. Zhu, welcome to the Neuron.

Speaker A: Thank you so much for the invitation and, uh, uh, I'm pretty honored. Thank you.

Speaker B: Excellent. Excellent.

Speaker C: Likewise. Likewise. Well, um, if you could, why don't you start with just giving us a little background on yourself and how you got into, um, doing what you're doing at Intel.

Speaker A: Yeah, sounds great. Um, so I grew up, uh, in China, um, so in the early days I have been, um, had been already obsessed with math. I think that's all everything started. And um, so my, um, PhD is also heavy math. It's on the computational science, um, side. So while I joined the Intel, I started with, you know, machine learning for design, basically design all kinds of algorithms, um, to accelerate the, you know, silicon platform designs. And in recent Years we started to broad out, uh, broaden up and to utilize machine learning and AI and latest uh, you know latest um ILP and all this for uh, you know day to day work and productivities and across all kinds of different professionals and the disciplines basically trying to scale the gen AI usages for intel devices and platforms and uh also for you know or across industry who really want to champion for local AI, ah hybrid AI usages for much more uh, economical and uh, much more private and a safer and sustainable AI future.

Speaker B: Wow.

Speaker C: Much needed.

Speaker B: Yeah. Could you talk us through a little bit? What's um, reaching its limit in the cloud first model of AI and why do you feel like hybrid computing is the most practical path for forward?

Speaker A: Yeah, that's a great question. AI is nothing but a piece of software, right? Just and all this, you know, software and all that it has to follow the physical rules and there are a lot of hard rules and govern it. You know um, number one is um the economically and uh, from environmentally. Right. So we um, we have spent so much tokens and every company, every individuals we burned so much token. It's not just coder even just Grant

Speaker C: and I oh we did a ah, we did a assessment of our company's ChatGPT usage and I think mine was like 800 million tokens for the month.

Speaker B: It was pretty terrible.

Speaker A: Yeah, yeah. You know like uh, you know even you just ask AI uh to write you a deep research report on some topics it can easily eat up your entire subscription within a few hours. So it's really hard, it's hard to uh, scale and to sustain for uh every company. And uh, another aspect is the uh data privacy, security. You know on the enterprise side it's very um, easy to understand. Right. It's uh, a lot of corporate secret data and customer information. Um on that you cannot just delay this, throw it into cloud and uh, where does it go? Nobody knows. Um, so I know a lot of the middle layer uh kind of agent companies and they said we have zero data policy meaning you know but it still goes somewhere right. So and it's um, that's why. And a lot of uh government policies, regulations in Europe and a lot of places and uh, that's, that's it makes AI adoption into different verticals become really hard.

Speaker B: Yeah.

Speaker A: Um, and um, even on the personal side right individual wise, you know there are so many you know fake videos and all that. I'm so concerned. I don't, I don't want to throw my daughters you know any pictures, videos and to, to do any of those, you know, those uh, AI alternations and things like that. It's. Yeah, right, yeah, it's concerning. It's really concerning. Um, and then on the um, you know on the other side is look at uh, you know if we all centralize this AI infrastructure in data center and how many data centers is do we need and the you know, the water cooling and all these infrastructures and how much impact even to our day to day life. Um, and I saw some articles and they, they are doing some in depth assessment of um, this massive, you know, data center impacting the neighborhood power grid and even impacting every single appliance lifetime.

Speaker C: Wow.

Speaker A: You know, Yara household and also the you know like, like I said, the water pollution, all that, it's, it's not sustainable. Um, another interesting aspect is for all these big AI ah companies, model makers and all that, they also struggle and they want to push their services to all kinds of users, enterprises but they are bottlenecked on um, this infrastructure they couldn't scale.

Speaker C: Right. I mean just look at anthropic and how they've had to deal with their troubled rollouts and all of a sudden you have fable and then you don't have fable. And part of the problem, suddenly growth

Speaker B: is the biggest problem you have.

Speaker C: But yeah, like you know, you have usage limits where you can't access it. Right. When you're getting to the point where you actually need it, you know, then you run out for the five hour or the weekly limit. It's like it's very difficult um, to navigate for everyone involved.

Speaker A: Yeah, exactly. That's why a lot of these, you uh, know, model vendors and all that, they start to apply, you know, read limit and the pick limit, all kinds of things. Yeah, yeah.

Speaker C: Oh yeah. If you want to use it for anything creative with video, it's so prohibitively expensive in my opinion.

Speaker B: It is, yeah it is. And I think you hit on the head of something here that we've chatted about a little bit before and that's that there really is a lot of incentive for companies to find ways to make this more efficient. Yeah, absolutely. Building data centers isn't cheap business. Like I'm certain that if it could be done with less, that would be the preference.

Speaker C: Yeah. Because then it makes the value of the data centers you do have that much more uh, valuable and you could certainly that many more customers too. So there is certainly efficiencies of scale.

Speaker B: Very interesting.

Speaker C: What's up? Humans managing virtual desktops across Microsoft environments can get Complicated fast. Nerdio manager for enterprises brings Azure, virtual desktop, Windows 365 and intune together into one console. So it teams can manage everything without bouncing between portals or stitching together point solutions. Nerdio also helps cut cloud waste automatically. Its intelligent auto scaling adjusts compute based on real usage right sizes, machines and can pre scale environments before users ever log in. So you're not paying for capacity that you don't need. And don't forget security and compliance are built in with automated policy enforcement and real time threat monitoring included as standard. And for organizations that are moving away from Citrix or other legacy VDI platforms, Nerdio automates much of the migration process to reduce manual work and downtime. More than 15,000 customers already use Nerdio from mid size to large enterprises. So get a demo of Nerdio manager for enterprise today@getnerdio.com theneuron and now back to our show.

Speaker B: So we've heard like versions of the mobile or edge versus cloud debate for years and years and years. What do you think has changed with generative and agentic AI? Uh, do you feel like there's a sense of urgency now that didn't exist before?

Speaker A: Yeah, I think everything is about timing, right. If you look at the history, everything is about timing every technology and all that. So uh, a lot of the technology like uh, Internet and the web, all that was invented started from cloud but gradually a lot of the modules like Java engines and all that started to um, running or partially running local for a lot of reasons, performance, privacy, latency, uh, uh, and a cost reason. So I think gradually it has to be a hybrid architecture and uh, it has to be partitioned in a reasonable way. Some of these pieces and running on cloud, some of these pieces running on edge. Edge. By edge I mean all kinds of AI devices. And now it's a booming or about the booming of I call it AI appliance. It's not only you know it's just like the form factor is changing and um, while was in China so a lot of interesting stuff AI, uh metrics and so it will talk to you and you know ban the higher ways you want to bend and during certain time it's basically embedding AI ah into all kinds of devices and then the computer and all that will, you know, will all evolve. So the intelligence has to be you know, distributed like we just talked about and uh, the seat where it needs to be. Right. So it cannot be all centralized in one place.

Speaker B: I agree.

Speaker C: Yeah, it's just not sustainable even as much as we would want it to be simple, it's not sustainable to do it that way. And also you mentioned AI devices. Well imagine if every device in your house was sending everything it hears over the cloud. You probably wouldn't trust that very much. So I think some uh, introspective uh, consumer would probably think well you know, I want some of my devices if I'm going to let them into my home to stay on the device. And I think that's one reason why we want edge devices.

Speaker A: Yeah. And also there's uh, another um, another perspective to this is the models capability. Just in recent uh, um, you know a few days or weeks there was a very, very popular chart Reddit, there's a hardcore community called local Llama. Yeah, you guys know this?

Speaker C: We know.

Speaker A: Oh yeah, yeah, yeah. So there's a famous chart and what it says is it really shows from how long does a model evolve from a frontier size of frontier model and to become a smaller size of model that can run on uh, laptop. So the analysis started from GPT3 generation and it took 37 months. Um, this model, the capability, the level of capability of this model actually came from, from cloud down to a laptop, um, and then analysis every generation GPT4 cloud is 3.5 and GPT5 and all that. So average it's about 24.8 months from tier running in the cloud to a small size model running on a consumer laptop. Yeah, consumer laptop.

Speaker C: Do you think that that capability, um, because you know it started at 37 months and then now it's to 24 if I'm understanding the chart correctly. Is that timeline going to shrink even further then do you think?

Speaker A: Yeah. So um, and just based on this popular discussion, right and what it says, if trends hold then fable miso level classes of uh, class of model uh capabilities could run on this high device at the consumer hardware. Probably within two years maybe it could be shorter. Yeah, it's pretty wild right? It's in two years and it's like uh, next year this time we can run quality 4 level of um, capabilities. High end laptop, right?

Speaker B: Yeah. Wow. Something I'm wondering is that like local AI is always sold through this kind of the privacy pitch is the idea like your data never leaves your device. But at the same time right now local models are generally less capable than frontier cloud models and that may continue to be the case for a while, at least for some amount of time. Maybe uh, depending on the news of the week. Uh, do you think that's a trade off? Users are willing to make a certain amount of intelligence for a certain amount of privacy and security.

Speaker A: Yeah, yeah, absolutely. That's a great question. So um, if you're running everything local and you get all these benefits, you control your data and privacy and all that. But like you said and event model has been progressing really fast but uh, today running on local devices we still see a uh delta between local uh and frontier models. Right. So but a user, you know why we use AI is we want AI to work for us.

Speaker C: Right, right.

Speaker A: And so the quality is important, you know. You know like say I'm not counting on my 9 years old daughter to do some serious stuff because the quality is not there. But AI is like this different size of models is kind of like uh, different similarities and the smaller model is more like a middle schooler and the frontier is like a college student type of. Yeah, right, yeah. But then how do we accelerate like a middle schooler to put it, put this AI to work and still deliver reasonable quality. And that's how we actually pair them together. They get mentors, they need supervisors, need a certain ways and distribute the right task. High schoolers could be really capable, really capable. If you can break down the task to a manageable size of task and uh, have very clear instructions they can do it. So I think that's fundamentally the philosophy of today's hybrid AI concept. You know and product we are rolling out is we do a you know we do this uh, um uh uh routing app uh or orchestration layer and really to uh look at for this specific tasks and ah. Is this uh simple enough or is this ah very complex. And then do we need leverage frontier models and to do some decompositions and then we allocated the right pieces to the local and to the cloud and then combine them together. Um so there are um a lot of aspects to it right. So first you know the first one is uh cost saving. So let's assume right uh, um we are doing some coding work and there's nothing super you know confidential and but still if you're running everything uh on um cloud um and it will cost uh a lot. That's why you have to offload some simple tasks for to local and uh then another aspect is if the work involves a lot of confidential data then how do we design a way of to ensure a confidential communication. It's like um, this supervisor teach the um small models and to look for the right information and put some structures to it but and the local model will never disclose any real data and local information to cloud. So those are the different considerations and uh, design aspect to this high risk version.

Speaker C: I feel like I'm already doing a version of this where I'm using Fable as my orchestrator and then I am having it like once it's figured out exactly what the task needs to be in a coding example, like to follow your example, I'm then having it uh, offload the work to a lesser model or in some cases a cheaper model like uh, Codex, which is not that much cheaper. Right. It's still frontier level, but I'm sending it off to Codex over the cloud and it's having it do some of the work to save some of my Fable budget. Because as we talked about earlier, you know, you have a limited usage that you have for the like top tier models. And so sometimes I'll be sending it off um, that way. But I know people who do that with local models and I think that that is a pattern that's very um, very repeatable. Um, and I think that uh, that should be what we should go for is like, yeah, you have the top tier intelligence that you can use and call on demand and then you can farm it out to smaller models or preferably local models that can do some of the stuff on your device.

Speaker A: Yes. And we saw some use cases that people even say after the task being um, decomposed basically. And a lot of this repetitive work, it's better to be run on local because you don't worry about the tokens and all that. And then you can run more experiments and to do a deep optimization to kind of scan through all kinds of different possibilities. Um, so we saw a lot of uh, user comments around that.

Speaker C: Nice.

Speaker B: I'm kind of wondering in the hybrid setup like you're talking about what things like specifically make the decision of whether this should be handled in the cloud or dinner. Because I assume this is happening in real time at some level.

Speaker C: Who's the traffic controller?

Speaker B: Yeah. Who's the air traffic controller? And like what sets its um, rules? Constitution of sorts to know what goes where and make sure everything's in the right Runway.

Speaker A: Yeah, yeah, love it. Question. Yes. So, um, that's the um, ox treater or router, you know, that's the design. Um, so across industry there are a lot of uh, efforts actually doing this research, um, model routing, um, um, you know, like Grant just mentioned, across even multiple frontier models. And which one do we use for certain tasks? That's on the frontier side and on the hybrid side is, you know, which one goes to the cloud model. You choose or you know which one goes to the local model you use and all that. So routing is a ah very big ah research area, very active. Um so currently um the approach we are taking uh is you know to really in a way is doing a classification meaning we look at this task and classify it into you know this is a um simpler task this local model can really handle and this is much more complex model and we needed the uh, we needed the ah bigger model um on PREM or cloud model to handle. But it, it's you know just um in principle it's, it's it's like that but a lot of uh. A lot of details, a lot of aspects to consider. For example like ah uh you know the um computationality. So if this query involves a lot of uh confidential information then where do you route and do you do PII information reduction then go to cloud or not. So those kind of considerations and ah um. So another uh, Another uh aspect of this uh routing is especially for local because the local could have a lot of um multitask, concurrent task uh going on. Then does it have currently at this moment have enough resources and to support this? Right. So a lot of um. This kind of uh, uh considerations are baked in and real time sensing all these changing factors and make the decisions.

Speaker C: Right.

Speaker B: Okay, so suppose you have this workflow built and items chained together. What happens when something breaks? There's a kink in the process. Uh is there a. It feels like this essentially chain and nest. If something breaks it could be a bigger problem than it would be when you're dealing direct with just one model.

Speaker A: Yeah. Actually what you're touching is um fundamental problem in AI. Um uh AI make mistakes and agents sometimes lie. M not um, sometimes lie all the time.

Speaker B: I had one being lazy this morning. I kept arguing with

Speaker A: exactly and he's like um, you know for example the, the execution plan is laid out. You know this, this agent has to call these tools go search that information whatsoever and then report it. I'm done. Success. But it never caught that too. It never caught that too. You know uh, it never did it. Yeah. Never get it. But it just lie back right at your face.

Speaker B: That was my conversation this morning. I was, I was having it assess our previous videos and it's like okay, there are 644 of them left to go. I'm going to start. And I come back and it's like they're all done and, and I looked in its traces and it's like looked at 32 videos and I was like hey, you didn't either and it's like. No, I sure didn't. You're right.

Speaker A: Exactly, exactly, exactly. It happens all the time. That's why, you know, like across industries, a lot of efforts going on with how to harden the agent harness, put the process, you know, audit and looking engineer, uh, a lot of work and to make sure agent is being monitored and being traced and doing the right things there. It's always, always you know this trajectory can be um, you know, can be, can uh, be logged and found and all that. So it's the same thing for the orchestrator or router design. And we have to make sure everything is being properly designed and you know, locked, audited and there's always a retry, fall back. So make the whole solution has to be robust, right?

Speaker C: Yes. So talk to us a little bit more about um, how you've been uh, implementing this with Superclaw. Um, am I right to assume that superclaw is built off of openclaw or is it parallel to openclaw? How do they relate to each other?

Speaker A: Yeah, actually um, so Silver Claw is uh, Intel's um, agent um platform and we aim to use it to help users help our customers and to um, you know to really uh, take advantage of AI agent workflows and also customize for themselves and utilize different modular IPs for their own designs. So that's the purpose of it. Um, so how we designed it is we utilized um, open code as the framework. It's not open clock. We chose it carefully across all these um, community solutions because it's more towards the production qualities, active support and all that. So we um, build on top of it and we did a lot of um, innovations and work around it and to make sure um, the deep optimization of the. Because open code is originally is still designed for cloud, right?

Speaker B: Yeah.

Speaker A: So they are more suitable for a much bigger frontier model. And once you want it to work for smaller models and you have to customize a lot of things and make sure you know that this harness really adapts to the smaller size of model capabilities. And we also add you know like um, um um, you know, governcy and uh, uh security, um, aspects of components and all that, you know to really ah, hardening the whole design and make sure it uh, uh delivers a very good experience. And we also um. Our aim is pretty high and we are not trying to say okay this is another community DIY thing. You getting it and you use it right. And we try to deliver a high quality so one user get it and uh, it's robust and it uh, give you high quality answers. So we use a lot of this cloud level of uh, um, benchmarks and data set and to uh, drive the development and the quality check and iterations of our software. So we achieved pretty good results like for example Pinchbench Office QAM router and all that. So I know there's a lot to improve but uh, I think we want to deliver very good experience and the bar is high.

Speaker C: I thought you were saying there's a lot of benchmarks and I was going to agree with you like, yeah, there

Speaker A: are, there sure are oceans of benchmarks actually.

Speaker B: That's interesting. Everybody has a benchmark.

Speaker C: How do you decide which benchmarks you're going to go after or which ones you care about? How do you decide?

Speaker A: Yeah, so um, actually because ah, not a single benchmark can really cover all kinds of usages. So we um, you know we tend to curate um, different uh, benchmarks and try uh, to do more comprehensive testings and optimization across different uh, benchmarks, cover different aspects. Um, uh, with the time limit. Yeah, that's important. With the time limit.

Speaker C: Um, fair.

Speaker A: And uh, we also comply with a lot of this uh, um, industry kind of like big trend, a lot of competitive uh, um, analysis. And so most of them converge to you know like one ah, top benchmark and then we use that as the final report, result, report out. Right. Because we want, and not only just report out result and we um, publish the kind uh, of testing scripts and all that to our customers, partners so that they can repeat it, repeat the testing, repeat the results. We just want the whole design to be uh, reliable, transparent and trustworthy.

Speaker C: Appreciate that. Yeah, that's awesome.

Speaker B: Yeah. So now that the first official public beta is out in the wild for Superclaw, what can someone meaningfully do with it today? And is there anything you hope the beta will be able to show or prove to you?

Speaker A: Yeah. So let me maybe quickly show you where you can find it.

Speaker C: Yeah, let's see it.

Speaker B: That'd be great.

Speaker A: Yeah. So this is the public web portal aibuilder.intel.com Perfect. Yeah. Um, this is the web portal uh, public is facing and everybody can uh, you know, access. So we put a lot of information out there, including our mission. So we really want to uh, deliver what you know, we discussed. Right. The cost, the trust and the sustainability. We want to enable users to utilize the intelligence, utilize the AI result, sacred filing, um, all that we just talked about. And then if you want to try it and you can go To Super Claw. And this is the dedicated downloading page. Um and we put the installers um and things like that right here and you can download and give it a try. And here are the different uh, um agents associated you know being incorporated into this release. And if you need more information around um, like uh, um testing script and config M and the modular designs and all that. And we have public beta, uh, public uh GitHub, um and it's also right here and you can utilize or you know browse through. So that's awesome. Yeah. Thank you.

Speaker C: You're welcome. Wait, so anyone. Because uh, is it totally open source? Am I right in saying that in an open beta like anyone can contribute to it or just anyone can see the code?

Speaker A: Uh yes, this is not completely open source but we open sourced part of it. Um and because the core algorithm being actively developed and we are also working with our customers partners um, you know the um, commercial pass, um so that's why it's not fully open sourced yet. But for um. Like I said uh, uh you know we openly share all these modular designs and IPs and all that for the, for the customers and everyone can build on top of it. It's your APIs. It has two aspects. The first aspect is it is directly user facing. So basically after you download it and you can immediately use it and uh. I'll tell you some interesting stories. We today in AAA as a developer team and we use it to find issues within our code base.

Speaker C: Love that.

Speaker A: Yeah we talked to it like okay, you look at your own code base. It's like you know we designed this whole thing. You know it's like a closing for you. Where do you feel like it fit, where it's not right. So we keep you know we keep uh, work with this agent super uh claw itself to improve itself and a lot of our developers also use it pair with different skills and to fetch the latest AI news and fetch the latest code change every morning. Wake up in the morning okay, what happened to code base who merged what? So there's a quick summary. So um, in a way our team we all just use as a uh day to day assistant today already for the development. That's one aspect. You download it, you can use it and plug into your code base and you can because the code base won't go anywhere. It's it. And then secondly it's very much developer friendly. So we purposely designed it in a way like each module is um, ah uh, uh plug and play. So like yeah we talk about the oxtreater router module itself is one ip. And so actually customers can use it to put plug into their own harnesses stack but help them to control the traffic. That's cool. Yeah, so and some other IPs as well. You know you use it as a sidecar microservices plugin just uh, as a ah, agent independent uh, uh, agent work for you?

Speaker C: Yeah. Um, I noticed also that you had like four different um, agents that you called out. You have the uh, you know, the deep research agent. You have an email agent, um, and a code agent. So um, for people who maybe aren't developers, they can use one of those other agents as their personal assistant like you just uh, recommended. And it's fairly straightforward for them to set that up. Could you maybe walk us through that process?

Speaker A: Yeah, yeah. So all those agents are kind of like uh, independent agents. You can pick one agent, talk to it. For example, the reason why we designed this email agent is we want it fully local. Right? Yeah, because it doesn't make sense to design a hybrid email agent like what goes to. Right. So that is purely local but uh, you can hook it up with um, right now we use uh, for the first one we enable the Gmail and you connect it and talk to it, ask your email, uh, you know, highlights or urgent items, things like that. So give you another example. For example this uh, you um, know, hybrid deep research agent, um, that is uh, another agent. Right. Again you can click to it and directly talk to it. And so this agent, uh, why we designed it is really, I think it solves a big, big problem, big charge for a lot of people. You know like we love you know, cloudy and proper plastic and ChatGPT and to do this deep uh, research for us, you know like you know, you have a topic and you want to find, you know, what's the best AI video editing tools and how to sequence them, what's the best price, how do I pair them together? Right. Probably that's what you are.

Speaker B: That's right.

Speaker A: And you want them to do deep, deep research. Not just the search for a second to give you an answer. You want all the comparisons and all that. Uh, and you want a very uh, in depth report. But sometimes this report need to be generated in a way by combining both local and cloud information together. Because if you are a CFO and you want to look at the, okay, the next quarter or next year market trajectory and your own company's data and how do we best positioning, right. How do we spend, where do we invest, you know Inevitably you need AI to look at your own data, which you cannot let a cloud AI to look at.

Speaker C: Yeah, yeah, right. Because even if you have a data policy with um, OpenAI or whatever, there's no guarantee there's somebody who reviews it for safety. That guy could potentially trade on that information. And if you're a public company, you know, the cat's out of the bag. I'm not saying that that's a hypothetical scenario. I'm not saying that actually happens. But you know what I mean, like there's, there's risks involved with doing this.

Speaker B: There's a lot of what ifs. Yeah, yes.

Speaker A: You know, like any CIO won't allow that. Right. So basically I don't know about your policy but for me right now, like I couldn't upload anything uh, to those cloud uh, um, AI engines.

Speaker B: It was that play for a while for us too.

Speaker C: But over time we've softened our stance, we've given in, we've given over to the machines.

Speaker A: It's just a strict block. You don't even try, you cannot. Um, but we still want those type of in depth report. How do we do it? It's a big question. Right. And then um, we carefully designed a process like I just mentioned and designed a way to doing this. Confidential communication between local and cloud. You know, like this is the high schooler and this is like the seasoned, you know, worker. And then um, without telling you know, all the data, sending all the data to the supervisor and the supervisor can draft out a plan. Okay. Like for this report and how many sections do we need and how much information do we need to gather and how much information we can find from cloud and the web search and all that. How much information we needed to find from local and we design schemas and all that so that you know, by iteration, cloud model will advise the smaller models and to fulfill the tasks. So it's a pretty clever design and we um, make sure there's no data leakage but we still deliver up to uh, um, like 90% of the quality compared to cloud only. Frontier model you think about best suppliers in the whole world who can do this deep research. Yeah, I think the number comparison was for office qa. I think the frontier, the best solution, best in class in industry is 76%. Um, and the hours is uh, 68 or 67, something like that. Yeah, so we, yeah we, I think those are like for the individual agent level and then it's viewed. Don't want to specifically pick any agent to directly talk to and you want default agent to route your different question and you could directly talk to the default agent. And uh, it has this agent obstrator behind it so it will dispatch the tasks to different uh, agents to do you know, to do web search and to do emails or deep risk. So that's another way of utilizing, using.

Speaker C: What I like about this is that you're basically creating systems to solve problems that you actually have. Like you're like hey, my company policy is I can't share any data to these uh, cloud models. So how can I create a deep research agent that still uses my data that I want to reference but then you know complies with my policy? It's awesome. I love it.

Speaker A: Yeah, yeah, yeah, yeah, exactly.

Speaker B: Something I'm curious about is um, when you look at benchmark results, talking about how local uh, processing can reduce cloud token consumption overall, at least for specific workloads, what should an enterprise or even a small business for that matter be measuring to determine whether hybrid AI is saving them money or just shifting costs into hardware and management instead of tokens?

Speaker A: Yeah, uh, that's a really good question because uh, for cloud low maintenance, use it, right? If you don't use it, stop it. But then for the investment of the local and you have to invest on um, the um hardware devices and that's a one time cost. But it's like everything else. This is like an overall trade off in life in many aspects. Right. You want to continue, you either pay

Speaker C: up front or you pay in which

Speaker B: bucket do I put my money.

Speaker A: But like we talk about if the trends holds true and uh, in a very near term and uh, individual household uh, and you uh know companies big, small and can actually comfortably host a, we call it intelligence hub. Right. And, and that is very fundamental transformation of everything we do today. And you uh know we'll have all kinds of AI appliance and devices running around in our household. But you need a intelligence hub to become the gateway. You cannot like you know, no matter anything, you just go to cloud. It doesn't make sense, right? You have to gateway solutions, direct the traffic, escalate to cloud or you know when needed and the similar things to you know like enterprises and companies, um, and uh, you know like it becomes m. I believe um, the future will be not like one general agent can serve all and then look at our app store, how many different thin slices, thin slices, vertical applications. We were talking about weather forecasting. I know we all have that but if you live in Florida you need a specific hurricane forecasting Or California where I live.

Speaker C: Right. Um, you need wildfire, you need watch duty.

Speaker A: Exactly. And today we still see a lot of this enterprises, users and all that haven't fully adopt AI and then bake all that into their day to day workflow. The reason is it's not specific enough and like you, you guys are doing video editing, there's no such thing as you like go do it agent. No, you have to hack, right, like uh, manually piece together many, many, many different AI tools and tweak and you spend a ton of time I play with it, I know the pain. So imagine uh, there will be a lot of this vertical agent and it truly, you know, you tell it what, what to do and you just need to you know more, you know, give, give the instruction around the taste right around the, you know, how it looks, feels and all that. And instead of like manually try to hook things together. Um, so I think there will be a booming ah, stage of such, you know, very much refined, polished vertical agents, vertical agent application coming to the world. So and that's right, yeah. With that then you think about like one department. This, you know, they may have 10 people, they do one thing and why do they need like a humongous ah, general AI to serve them. They just need that you know, one specific vertical agent and that can be capsulized mostly uh, devices and serve for 10 people and another department doing something else. And that's a different thing.

Speaker B: And most of those don't even need a state of the art model. The truth is the vast majority of work today can be done with much, much smaller models. Like when you think of it in terms of breaking things down into tasks and looking at the different parts you could do smarter resource allocation could save so much money and so much rate

Speaker C: limit for uh, I will say like the tricky part for me personally is you just don't want the agent that you use to fail at the task. You don't want hallucinations, you don't want it to be too stupid to get it done. So that's why we default to the higher intelligence model today. But once you can trust that the smaller models are you know, intelligent equivalent, um, once we solve some of these edge cases then yeah, you're right, there's going to be an explosion of vertical models I think.

Speaker A: Yeah. And you know the other day I met a brilliant professor and uh, now he's uh, a startup, uh, founder. So his focus is all about cognitive learning, you know, about intel, how to build intelligence. It's so our conversation is around Just like what you said like today actually. Look at all this frontier model. The intelligence level is pretty high, right? Yeah, yeah, it's really high and has

Speaker B: been for several generations even.

Speaker A: Yeah, but then why can't we put it, put, put it to work. Serious work.

Speaker B: Yeah.

Speaker A: And so uh, I'm thinking about analogy and let's see if you agree. I feel like it's like we have college students graduate but they haven't been through all these serious trainings for different specific domains. They have no domain knowledge and no hands on practice and um, no specific know hows and all that. So that's the last mile. And how do we train college students into experienced worker and fast. And that's the last step of put agents to real work. And that's the domain specific and how do we get there? And a continuous learning.

Speaker C: So that uh, totally everything that you said I think is 100% true. And I think we've seen with the new models that come out that they can only focus on so much. Right. You can't specialize them too much in one area because of the architecture. It will have this thing called catastrophic forgetting where it will forget the good things that it learned early on in its training if it specializes too far in any one direction. So it's sort of like that uh, what is it a radar chart where it, it like, you know, you can have all of these like um, lines going off towards different um, specifications in a circle. But, but you can't go too far in any one direction or it'll like lose some of the skills from another direction. So for example a lot of them are really good at coding now, but they're not as good at writing. And we think they haven't been good at writing the frontier level ones for like a little while. Ever since they've been focused on coding and everyone's like what's this about? And you know, I think it has to do with this. Like you go too far one direction, you can't, you know, you can't specialize enough uh, to be specific.

Speaker A: So yeah, and has to also back into like this domain specific experience because they have specific practice and workflows and all those practices and uh, like you said, how to make sure agents do not lie and really deliver to the quality and to the promise. I think all the other trainings um, has to be, has to happen in the next one, two, three years because that determines in my opinion whether AI will go big or it will go the other way around. Because the big promise of AI is greatly enhance humans productivity. In simple terms, they have to work. Do real work. Right. Do real work. So um, I think we are in this mission and to make these devices, make the building platforms available so that we can have more developers can utilize such devices platforms and to build this vertical agent and truly make A.I. um, useful, reliable. Right. Um, and carry on a lot of repetitive works, work you know, for different industry verticals and fundamentally improving the productivity

Speaker C: and uh, uh, doing it sustainably, you know, not burning. Yeah, not burning a hundred, you know. So all these tokens, you know, over the cloud.

Speaker B: Yeah.

Speaker A: Yes, yes, yes.

Speaker B: Well Dr. Zhu, thank you so much for joining us today. It's been a lot of fun.

Speaker A: Thank you. Thank you so much you guys. Um, you know, it's just so easy to talk to you and I feel like it's just we are having a coffee chat. Um, it's a lot of fun and especially you guys are very much hands on, uh, AI. So we are like a click, you know, just like that. Right.

Speaker C: Well, you'll have to come hang out more often.

Speaker B: We'll have to do it.

Speaker A: Yeah, yeah, yeah.

Speaker B: Where can viewers go to try Superclog in and check out your intel hybrid work.

Speaker A: Aibuilder.intel.com okay, perfect.

Speaker B: Excellent. Excellent. Well, thanks again. If you liked today's video, please take a moment to like and subscribe. Don't forget to check out the Neuron's other projects, including our daily newsletter read by more than 700,000 people just like you. Uh, also the Neuron Academy, where you can learn all about AI and how to use it in your work and life, as well as our newest sister newsletter, Robotics Insider. That's all we have for today. Thanks for joining us. Farewell for now, humans.