The InfoQ Podcast · 2026-10-30 · 1h 11m
Key moments - from our scoring
Substance score
41 / 100
Five dimensions, 20 points each
The 2026 AI and ML trends report reveals a fundamental maturation in how organizations approach AI systems. Rather than chasing the largest frontier models, enterprises are increasingly matching specific models or combinations of models to individual workloads - a shift that moves AI from a model selection problem to a system design problem. Chip Huan, Arthur, Hien Lu, and Daniel Dominguez discuss how model distillation, quantization (down to 1-2 bit models), and the rise of smaller language models are enabling faster, cheaper inference without sacrificing performance for most use cases. The conversation highlights that 2026 marks the transition from pure interaction-based AI (asking questions, getting answers) to delegation-based AI, where agents plan, execute actions, and operate autonomously over extended periods. The panel emphasizes that successful AI adoption now depends less on model capability alone and more on operationalizing AI in hybrid environments - balancing on-premises and cloud deployments for data privacy, compliance, and cost optimization. Type Safe AI's JAV models are emerging as a new class of extremely fast, low-cost models optimized for software outputs rather than human consumption, while the year is being dubbed "the year of inference" as organizations focus on deployment, optimization frameworks like vLLM and SGLang, and running open-weight models internally for compliance-sensitive domains like finance and healthcare.
Hosted by InfoQ. The show scores 66.5/100 on substance across its analysed episodes. See the full show profile.
Frontier models continue advancing in reasoning, coding, and multimodal capabilities, but the trend is toward matching specific models or combinations to individual workloads rather than selecting one universal model. Smaller models, achieved through distillation and quantization, now deliver comparable performance at much lower cost and latency, making them preferable for most production use cases.
AI is shifting from interaction-based (users ask questions, AI responds) to delegation-based systems where agents can plan, use tools, execute actions autonomously, and continue working over extended periods, fundamentally changing how AI systems are designed and operated.
Model distillation trains a smaller model to imitate outputs from a larger, high-performing model, achieving comparable performance at much lower computational cost. Recent advances in quantization and distillation techniques enable significant reductions in model size (even to 1-2 bit models) with minimal performance sacrifice.
On-premises deployment of open-weight models addresses data privacy, compliance, and PII handling requirements - particularly in regulated industries like finance and healthcare - while also reducing inference costs compared to cloud-based frontier models.
JAV models represent a new class of extremely fast, low-cost models optimized to output software (not human-readable text), unlocking new use cases like UI building and software-to-software interactions at significantly lower latency and cost than traditional LLMs.
Our reviewer’s read on each dimension, with quotes from the episode.
There are genuine insights scattered throughout - notably on context pollution worsening with larger windows, the backtracking problem unique to robotics vs. LLM training, and the multi-agent task-forwarding failure mode - but they're buried in extended filler, generic excitement statements, and housekeeping that consumes significant runtime. Insight-per-minute ratio is low.
Context pollution is actually the biggest problem...more context can make an agent slower, more expensive, less reliable
one agent started forwarding tasks to the other agent and then that agent forward to another agent and then none of them did the work
The episode recycles widely-circulated frames ('year of agents,' 'AI is moving from interaction to delegation,' 'multi-cloud for AI will mirror what happened with cloud') without attribution or challenge. The few genuinely fresh moments - Arthur's context pollution argument, Chip's first-person multi-agent failure anecdote, and the JAV/TypeSafe model framing - are not developed far enough to count as original theses.
the model itself is becoming less of the whole story...we're changing the question from which model is best to which model or combination of models gives us the right capability
Every AI agent will incorporate some form of the Jeff model into the AI agents
Chip Huyen is a genuinely high-caliber practitioner - author of 'AI Engineering,' active in AI systems work - and brings the episode's sharpest concrete observations. The rest of the panel is composed largely of InfoQ/QCon editors and a vaguely described researcher-consultant, limiting the practitioner depth that would elevate this score further.
I'm Chip...I'm working on a lot of AI stuff. AI agent tech, getting AI to work well for like workflow
I'm Daniel Dominguez. I'm the deputy editor at InfoQ
The episode names real companies (DoorDash, LinkedIn, Harvey, Physical Intelligence, Cursor, Replit) and protocols (MCP, A2A, FIPA), which is a positive signal, but almost no actual metrics, benchmark numbers, dollar figures, or timelines are offered. Chip's robotics section mentions 'billion dollars' without specifics, and fine-tuning and inference claims are made entirely without supporting data.
If you look at example from DoorDash, they build an AI systems for ordering food
I'm sure they do fine tuning there. But beyond that I wonder if enterprises, they actually do it
The host largely asks open 'what are you seeing' questions and rarely follows up or challenges a claim; transitions are frequently just 'thanks, let's move on.' The most incisive questioning actually comes from Chip as a panelist - probing multi-agent orchestration patterns and sparking the best exchange of the episode - not from the host.
When you mention multi agent system, which pattern usually go for? So is this like one orchestrator is a spin up, like multiple workers?
has any of your company like fine tuning the model Artur mentioned? It's like waiting for budget. It's supposed to be expensive
Computed from the transcript - who did the talking, and the words that came up most.
In this episode of the podcast, members of the InfoQ AI and MIL editorial staff and friends of InfoQ discuss the current trends in the domain of AI, ML and Data Engineering. One of the regular features of InfoQ are the trends reports, which each focus on a different aspect of software development. These reports provide the InfoQ readers and listeners with a high-level overview of the topics to pay attention to this year. InfoQ AI, ML and Data Engineering editorial team met with an external guest to discuss the 2026 trends in AI and ML areas, and what to watch out for the next 12 months. In addition to the written report and trends graph, this podcast provides a recording of a discussion where expert panelists discuss how innovative AI technologies are disrupting the industry. Read a transcript of this interview: Newsletter:
Transcribed and scored by The B2B Podcast Index.
Speaker A: The decisions you're making right now about AI adoption, architecture trade offs and how your team works together will shape your systems for years. Getting those calls right when the landscape is shifting this Fast is hard. QCON San Francisco has spent 20 years connecting senior engineers with practitioners who are a few steps ahead on the same problems. This November 16th through the 20th, 60 plus speakers across 12 tracks will share what's actually working in production and what isn't. No hidden product pitches, just senior practitioners helping senior practitioners learn more@ah.qconsf.com.
Speaker B: Hello everyone. Welcome to the 2026 AI and ML Trends Report Broadcast. Greetings from the InfoQ AI, ML and Data Engineering team. We also have a special guest today for this year's Trend Report discussion. This podcast is part of our annual report to share with our listeners what's happening in the areas of, uh, AI and ML technologies. I am Srini Pinchikala. I serve as the lead editor for the AI, ML and data engineering community on InfoQ. I will be facilitating our conversation today. We have an excellent panel with the subject matter experts and practitioners from different specializations in the AI and ML space. I will go around our virtual room and ask the panelists to introduce themselves. We will start with our special guest first, Chip Huan. Hi Chip. Thank you for joining us and uh, participating in this podcast. Could you introduce yourself and tell our listeners what you have been working on?
Speaker C: Hello, I'm Chip. I'm very excited to be here today. Uh, it's an honor. I know that everyone on the panel has amazing background. I've been following the world for a long time. Today I was hoping to share what I know, but also hoping to learn from y'. All. I'm working on a lot of AI stuff. AI, uh, agent tech, getting AI to work well for like workflow. I'm so very interested in making AI just like faster and cheaper. We saw recently the typesave AI that came out. I think there's a huge demand for cheap and fast intelligence and that's very exciting.
Speaker B: Thank you chipiya. Definitely. If we have time, I would love to learn more about this Java model from Type cfai if you have any experience on that. Daniel, how about you?
Speaker D: Hi, I'm um, Daniel Dominguez. I'm um, the deputy editor at InfoQ. I focus on, uh, artificial intelligence, machine learning and data engineering. I'm also a software engineer and lately I've been particularly interested in AI agents, agentic software development and how AI is moving from purely digital system into robotics and physical AI.
Speaker B: Thanks, Daniel. Arthur.
Speaker E: Hi everyone. I'm Arthur. I'm a researcher and um, technology strategist basically and these days I split my time between consulting advising companies about how to make AI useful in your systems and um, apply research basically on um, multi agent systems and belief revision.
Speaker B: Thank you. And uh, Hien. Uh,
Speaker F: hi everyone, my name is Hing Lu. I mainly focus on ML infrastructures as well as I guess nowadays agentic engineering. Currently I'm helping with a startup on Consulting on AI inference and a long term uh, fan of QCon, also an editor of InfoQ, also a uh, QCon program chair for QCon AI New York.
Speaker B: Yeah, thanks. Yeah, definitely. I very excited to talk with you all about this topic because uh, you have been part of uh, InfoQ and QCon family for so long. It's always great to chat on these topics. So I'm looking forward to speaking with you about what's happening in the AI and ML space where we currently are, but more importantly where we are going. Right, uh, it's a fast pace as the AI, ah, technology innovations keep happening. Uh, since we discussed this report last year, before we start with the actual podcast topics, I want to go over a couple of quick housekeeping items for our audience. There are two major components to these 10 reports. One is obviously this podcast, which is an opportunity for you to listen to the panel of expert practitioners and uh, see how the uh, AI innovations are disrupting uh, the industry. The second part of the trend report is a written article that Arthur also mentioned. We can cover a lot more topics in that article so that will be available on InfoQ website along with the podcast recording. That's where we actually assess the current uh, technology adoption. And we have an adoption graph that we update every year to uh, highlight which technologies are kind of further ahead in the adoption cycle than the other technologies. It's always a great reference uh, for anybody who is following the technology innovations in the AI space. So definitely look for those two parts, the podcast as well as article. And also I definitely uh, recommend to you to check out all the Info Q publications in this podcast. We will uh, definitely discuss the important developments in AI technologies that took place since our last report back in August last year. But due to a limited time in this forum, we won't be able to cover all the topics or even get into in depth discussions on some of the topics. But we will try to follow up on those important topics as separate podcasts or separate articles because they are important for us to learn about. But again, I encourage you to check out the InfoQ website and AI and ML and data engineering, uh, community, which includes news items, articles, podcasts like this and presentations. So it's also worth mentioning m. You know, with all the AI stuff happening, that this podcast is not AI generated or even AI assisted. It is being recorded with real people who have real world experience. I just want to get that disclaimer out of the way. Right. So contrary to all the predictions being made out there that AI is going to replace everything. So that being said, let's get started. It's been almost four years since ChatGPT, uh, came out right back in November 2022. It's been two to three years since Rag Technologies kind of started seeing uh, enterprise adoption and autonomous LLM agents came out. So definitely a lot of things have been happening. Agent AI has been the kind of main dominant topic for a long time now. So we'll definitely look at that a uh, little bit today we are seeing more developments in the quality assurance space with the AI evaluations and harness frameworks and all the other uh, uh, good topics. We, we'll get into more of those details later. But first topic is I would like to get a kind of quick uh, take maybe 30 seconds take from each of the panelists on how do you see the current state of AI in 2026? Chip, could you start us off?
Speaker C: So it's going absolutely crazy. So I'm tracking like on the money in and out of like AI company. So money in as in like private investment and then money out, acquisitions and ipo. Right. So for, for many, many years you see the charts like very similar, like when the three numbers. Because the money in has to come out at some point. But in recent years as I go like this, the money in is like go like exponentially and the money out, there's no money out at all. It's very exciting. Has the amount of like money and it's like I think proportional to like I guess I use as a proxy for people excitement. A lot of companies the number of company size is going down. Is the average like median company size going down significantly. So that means that like people can do a lot of things that anyone can build nowadays anyone can have a product. So the issue is not really about like how to build things anymore. Like anyone can build things, right? But like what you build and then like how do we as a receiving end like filter through all of that is quite insane. And I think I noticed going through these things like winner catch all situations because like a few products somehow like Surface and everyone just talk about them and like the rest is like lost in the shuffle and it's like, very, very tricky. It's just like everyone's like force fest, the same kind, like ideas, and at the same time we have a lot of new things coming out, but somehow everyone is talking about the same thing. They don't get it. Yeah, it's very, very strange.
Speaker B: How would you hear what are you seeing as a AI, you know, uh, in 2026?
Speaker F: Yeah, obviously, like Chip said, there's a lot of excitements, innovations, product announcements. But I have to. If you want to talk about like, at the beginning of like 2026 or the end of 2025, you're talking about the year of agents. I think we are in a phase where the AI agent stacks. It's, it's being built in a way, the different layers. There's a lot of progress there, a lot of exciting, a lot of innovations for all of us, how to learn how to build AI agents. And now everybody's trying to figure out how to operate them, how to bring them to production. So that's very exciting space from an enterprise perspective. I think the other one that I'm sure everybody is aware of is writing code. It's easy now with coding agents. So I think there's, uh, many exciting ideas going to be built and we all need to adjust in terms of how we build software moving forward. So I think that's another area that's happening in 2026, and we'll definitely see more excitements in 2027.
Speaker B: I think we'll get into those topics a little bit more here later anyway, so we'll have time. So how about you, Arthur? What do you see at, uh, a high level?
Speaker E: Yeah, I think follow up. Um, I think what we're seeing now is we're changing or shifting our perspective from the parts of the car to the car itself. I mean, you have impressive isolate capabilities in each model being developed year after year. And now you're starting to realize as companies in production that we need them to work and to be dependable. So, I mean, you can have a pilot of a great agent that can work beautifully, can have curated data, and that's great. But then a good prediction that have. Your data is messy. You have different permissions, you have regulations, you have. The system has to be affordable, it has to be recoverable. So that feels like the defining test for AI that we're seeing emerging right now is not whether it can do something once, but what I think can trust it in a real workflow to solve Real problems. So that's when you see it actually being applied in companies and being successful.
Speaker B: Thanks, Alter. How about you Daniel? Uh, what are you seeing?
Speaker D: I think the biggest change in 2026 is that AI is moving from being something that we interact to something that we delegate. Like hen was saying, I think uh, most AI applications over the last few years were based on a simple interaction. You just ask a question and the mobile gives you an answer. Right now we're starting this year, we are starting to see system that can plan, can use tools, can execute actions, can do their own work and continue working for much longer period of time. So I think uh, this year is less about another jump in the model intelligence and more about turning that intelligence into actual systems.
Speaker B: Thanks Daniel. Thank you for that. Yeah, we can uh, that's a good kind of segue from there. Right. With AI agents and all the other, uh, innovations happening, the foundation is still on those models, language, uh, models. So maybe that's where we can start our kind of main discussion today because there's always a lot of uh, new developments happening there. So in 2026, my uh, observation is the language models have shifted from these massive size models, billions of parameters, into the smaller size models, small language models, where they can uh, be faster in terms of performance, may uh, not be as accurate, but uh, accurate enough for most of the use cases. And also we are seeing the rise of reasoning models that can think through math and logic problems before actually responding. Right. So a lot of good stuff happening. So we can uh, start with this as our main topic. Uh, Arthur, I know you've been um, looking at some language model topics and others frontier models. So what do you see is happening here? What our uh, listeners should be aware of or should be learning about?
Speaker E: Yeah, I think in general, uh, when you focus on specific models, uh, we have better reasoning, have better coding, have better multimodal abilities. They're emerging node, it's evolving. Right. But then we're going to go back to what I said first, that if I were to pick a headline for frontier models in 2026, I'd say that the model itself is becoming less of the whole story. So you do have the capability. You do have to worry about it, of course. But we're changing the question from which model is best to which model or combination of models gives us the right capability, latency, data boundary cost for this workload, specifically this workload, which to me at least the trend seems to be a move from choosing one model to deliver matching models to workloads which makes it a uh, system design question, not a model choosing question. Right. And I see that a sign that the field is maturing because the model stops being the whole problem and becomes the one component of the architecture. So we do have to worry about all the specific capabilities of each model, but they have to work in unison together.
Speaker B: Any other thoughts uh, from anyone else on the models?
Speaker C: I think model size is a little bit interesting because the definition has changed over time because the model size, number of parameters, it also depends on how many of them actually active at the same time. It's a trillion parameters model, but only a few percentage. Also another is the quantizations. I do think it's just very exciting work on quantization. Recently I feel uh, about two, three years ago people thought it was crazy to have a four bit model, but now they have consistently see people running like one two bit model which is like a significant reduction in size. And we think of like as a compute requirement or the compute efficiency of these models. So I think it's interesting to see another trend I see from models is that of like uh, release cycle. Because like, let's say that you're Frontier Lab, you have a very, very strong model, right? You release this out in the top of every benchmark and then a few weeks later you will see that release another model much cheaper, smaller, almost like equivalent, but maybe 1 or 2% drop in performance. So at MX this is usually a trend of like yes, we have a big model and then you distill that into like a lot of other techniques. Uh, like you think a bit of a mixture of expert, making it more sparse, quantization aggressively quantize into like smaller and a cheaper model.
Speaker F: Yeah, there was a lot of interest in like doing fine tuning toward the beginning of the year and there's a lot of innovation in that space with lore and all that stuff like that. And all these kind of neo cloud inference providers started providing the ability to do fine tuning for at least the open source models. I would love to hear from the panel if you have heard successful use cases in the enterprises, they actually do fine tuning for their specific use cases. I think there's some happening but I don't hear much about it. Obviously it takes a lot of effort to do that with the right skill sets. Maintenance, operations, doing inference. The whole sherman is doable but whether that is make you reliable and in operations and all that stuff like that is still a lot of work. So maybe things will be easier in 2027 and people do it more. But that's still a question in my mind of how pervasive that is being actually done in enterprises.
Speaker B: That's a good point to hear. Like how do you make the model management lifecycle kind of similar to software development lifecycle. Right. How do we make it repeatable and streamlined? Yeah,
Speaker F: exactly. Yeah. Unlike you know, traditional ML, you start with data and then you train and you evaluation and you bring it online and here you do fine tuning. So there's still, it's a lot of work but I'm sure there are specific use cases, particularly perhaps these AI native companies in the domain of legal. Right, Harvey. And such. I'm sure they do fine tuning there. But beyond that I wonder if enterprises, they actually do it. That's kind of my question.
Speaker E: I've seen a couple of companies doing that but they're very specialized in what they do and ah, we're very, very careful about uh, ip so they won't disclose that for a few years at least I think. But usually I'm seeing this basically in healthcare and finance. It's concentrated there for some reason, you know the reason. But yeah, it's happening. Just need the budget. But it's already happening. It's great.
Speaker F: Makes sense.
Speaker C: Yeah, it was just about to ask like, has any of your company like fine tuning the model Artur mentioned? It's like waiting for budget. It's supposed to be expensive.
Speaker E: I think it depends on what you're doing, how you're doing. For example, when you come to open models, uh, I've seen a small rush of uh, open models and local models and people thinking that okay, it's going to cheaper but think about open. When I have the word open, you have open weights, open license, transparent training data. I have a lot of things involved and they don't necessarily mean lower costs or commercial license or whatever. So depending on what you're doing, you need uh, massive data or you want to get traced out of big models to train smaller models. Thing is, I think it's still more of a core craft, the artisan thing, the process, you know. So people are discovering this and although you have the process for training models and for training things, applying that to your real world problems is still a process of discovery. So they end up spending a lot of money. I think that that comes naturally. It's R and D basically.
Speaker B: Yeah. Hopefully some of these uh, learnings from these companies will be shared with the community next year, the next couple years as Andre mentioned. Yeah, I want to kind of maybe uh, talk a little bit about the model, um Distillation topic. Uh, Chip, I know you talk about this in your book titled AI Engineering. I highly recommend everybody to check out the book by Chip, uh, one of the very, um, valuable books in the AI space. Uh, so you talk about model distillation. Can you uh, discuss like how it works and how it can help the overall model usage?
Speaker C: Yeah. So I think model distillation is extremely important. It's very exciting. And I think a lot of us, uh, today benefit from model distillations either done by some providers or some application developers. So idea is that you have a very big model, it performs really, really well. But of course these big models are very expensive to run and of course a lot of them are very slow. So can we just get another smaller model to imitate a load from these big models so that it can achieve comparable, almost comparable performance but a much cheaper price. So a lot of time you can get like. So we care about uh, something called like compute efficiency. So we easily think about, okay, like for this much drop in the compute efficiency, like how much is the performance we have to sacrifice. And this has, magic has gotten going like really, really, really well in recent years. I think people are aggressively like developing a lot of new techniques and it's working things didn't think would be possible even just like a few years ago.
Speaker B: Uh, also in the models we have local versus on the cloud models, right? We can run um, some of these models locally using Ollama or uh, the other, uh, LLM engines. And also there is a multimodal models like, you know, for the text and images. So anybody have any comments on those, uh, topics like running these models on prem versus cloud or the multimodal models?
Speaker E: I think regardless of the objective of defining what you're using, you need to separate a few questions what we're doing because have you seen people defining on prem versus cloud as a full thing or really separate, really binary thing? Okay. I mean it depends on where the client runs, where the model runs, where the prompts, tool outputs, logs go, who keeps them, Everything's involved, right? So in the end, both setups tend towards a hybrid approach. They're neither, uh, fully open the cloud. And thing is why is it important to define that for the company? What they're trying to solve is this a budget problem, is this uh, architecture problem. So I mean, okay, you can categorize them, but what for what we're doing with it? So which questions are you asking about this? It's not as simple as choosing a category and that's it.
Speaker B: Thanks Agar. Yeah, definitely a lot of other stuff happening. One thing I just want to mention briefly, uh, about the JAV models, right. These are the, what they call System one models. I just started learning about these. You know I have no in depth knowledge on this, but yeah, I think we will definitely share more about this in the future. But uh, has anybody else done any reading on this? Like this uh, JAV model from Type Safe AI that they talk about, you know like very fast inference and very low cost compared to the LLMs.
Speaker E: Yeah, it's supposed to be my weekend uh, experiment for
Speaker B: Cool, cool
Speaker E: tomorrow.
Speaker F: I've been watching um, a lot of videos on this and recently I watched a podcast from Linked, uh, space where they interviewed the founder of Type Safe. Very, very fascinating piece of technology. One of the key things I took away from that is a different type of model, but it's also extremely fast. I think that's going to help unlock an interesting new set of use cases. There are a bunch of interesting demos about using that kind of decision to build UI and other use cases that you might have not considered. Like hey, is that a relevant use case? But it's a lot of us to learn and figure out how to incorporate this. But I think the key idea is, at least from the founder is what he want to do is to build a model that is for builders and the output is for software, not for human. Right? That's one of the key things he pointed out.
Speaker B: It's uh, basically evaluates the state and returns the answers. It's mainly for the software. So we can maybe uh, hear. You mentioned about inference. I heard the same thing from my own um, experience. So 2026 is being dubbed as the year of inference. Right. So could you talk about like, you know, how we are shifting from training models to running AI inference and you know, what does all this mean in the real world usage? What are you seeing in terms of inference frameworks?
Speaker F: Totally. The inference topics, I mean exploded since the beginning 2026. Obviously from one end you have all these inference stack, right from vlm, um, sglang and so on that are helping uh, the rest of us to spin up our own inference service and our own environment. And at the same time these open weight models are becoming more and more capable and that space is like initially not a whole lot of people how it works and how to operate it, but now there's a lot more information, some resources out there about how to actually use these software, how to optimize uh, them and at the same time some of the Use cases. We talk about the you know, financial companies where you have PIIs. So the ability to host these open waste model internally for those kind of specific use cases are ah, very relevant and I believe that that's going to continue to grow, continue to be the case where company will do more of this moving forward. Now that is a lot more approachable, more easier to do. There are some companies that because they want to save cost they actually want to host internal open wage models for coding for example. So for many reasons that inference going to continue to be a very, very useful, very popular topic and more and more companies will start doing of this moving forward in my mind.
Speaker C: Yeah, it's very exciting. Like for the work um, on inference do you have like focus on specific chips or like hardware architecture to inference on?
Speaker F: Not so much at this point because AMD just coming online but more and more inference provider at least the NEO cloud they want to be able to do inference across chips. Nvidia and AMDs and potentially near in the future Intels and so on. But at the moment still predominantly Nvidia uh, from um, or what I gather.
Speaker C: I'm curious because I think it's just a trend of as you get like newer, newer chips, right More powerful chips. There's many, many, many older generation of chips, right and somebody like they like almost come for free because I love functional, don't use them. So like what, what do we do with them? So I think like getting those done on like older generation of chips, like making things faster could be very interesting like having like smaller. I think it tied like model installations. They got to have a smaller cheaper models that require less compute footprint. You can fit them onto the older generation of chips and also like a lot of like upcomers for like Chip because it's very hard to compete Nvidia on the, on the like on um, the flagship architecture. But you can definitely compete at like a lower tier architecture.
Speaker B: Yeah, that's probably the next uh area of the competition. Right. I heard companies like Nvidia and Apple are definitely dominating the, the hardware, you know, space. Right. You know in terms of demand.
Speaker F: Cerebras is an interesting chip player and I think we'll see more of that maybe later on this year.
Speaker B: Ian and Chip, you mentioned in a previous conversation about agentic loops and hardness, uh, engineering and context engineering. Can you share maybe starting with Chip, uh, you know, can you share like what's uh, what do you see happening in the context engineering space?
Speaker C: I do think definitely like in uh, in the last couple of Years agentic like coding use case have it dominate everything. But I think like with a typeset launch it shows that like actually like people have been so focused on coding, there's a huge tap area for like non coding use cases. For me, like coding is definitely very important, but I definitely see it realize it's like it's not the only thing.
Speaker B: What are your thoughts?
Speaker E: Context pollution is actually the biggest problem, the problem worth emphasizing now because uh, we've seen the past year or so movements toward bigger context windows in general. But they don't really solve the issue if the system cannot distinguish what's useful, what's current, what's alternative information from irrelevant, stale or virtual material. So I'm bringing something from research here, but in general, more context can make an agent slower, more expensive, less reliable. So the practical question would be can, if you have your context window, regardless of the size of it, what enters the context from which source, with which permission, uh, for what step of the workflow. So you don't have this huge pollution. It touches topics on M retrieval, quality, source distribution, precious rules, context model in general, and context compression. So I mean there is a huge, huge heap of problems underneath the word context. That's how I see it.
Speaker B: Thanks for seeing that. So yeah, in the interest of time, I do want to kind of move to the topics. I want to focus on the agents and the evals a little bit more in this, uh, discussion. So we can jump into the uh, agents topic. Daniel, I know you've been uh, looking into this space, you know, for a while now. So this continues to be one of the major trends in the AI space. Agents, you know, they have gone from, like you mentioned earlier, from like just coding assistance. And you know, now they uh, can act on several different uh, aspects.
Speaker D: A. Agents are probably one of the biggest trends of 2026. What I find interesting is that agents are moving through several levels very quickly. So we started with personal agents helping one person. Then we start seeing agents working with teams. And now companies are thinking about agents operating across entire organizations. So I think the important change is that we're moving forward from beyond just a chatbot model. Instead we are simply generating an answer. An agent can have goals, can use tools, can interact with software, can make decisions, can keep working until the task is complete. So I think that changes AI from interface into something closer to an execution layer. We can see this mainly in software development. Software development is probably where this transition is more visible. So we move from autocomplete to copilot and now we have coding agents, we have tools like Cloth, Codex, Cursor, Replit and others that can navigate repositories, can modify multiple files, can run commands, can execute tasks, can inspect failures and can iterate on their own. So the interesting question is no longer can AI write code? We already know that AI can write code. I think the more interesting question is how much of the software development loop can we delegate to an agent while we still maintain quality, security and control? That is starting to change the developer's job. So I think uh, developers are spending less time producing every line of code and there is uh, more time defining to intent, giving context, reviewing decisions and validating what agents can produce. So I think the recent developers surveys also show how quickly agent coding has become part of everyday development workflows. So I think uh, there are a lot of discussion right now on the interface, disappearing the agents and enterprise integration, multi agent systems, all the challenges that is bringing. So definitely this is something that we, we're going to continue seeing probably next year and a couple of next years all this agent and agentic phase working across organization and on a daily workflow of not only developers but in other areas of the work.
Speaker B: Yeah, thanks Daniel. I think uh, as the AI tools generate more and more code, I don't think that will be any different than humans doing it. Maybe they will be faster but in terms of the defects I think I'm sure the code will not be perfect right off the bat. So how do we see these generated code? How do we make sure that it's safe and secure and uh, in compliance with the corporate standards? Uh Arthur, how do we safely connect AI agents into the software development platform under SDLC process?
Speaker E: That's a big question, isn't it? First we need to separate coding assistance from coding agents. Assistance would be something like an agent helping uh, a developer navigating a code base, writing a test, drafting a patch, something like that. Agents on the other hand means assistant can run comments, modify repository, open pull requests, affect deployment in any way. And for that we need operation design. So for operation design you need all the steps, you need authority levels, permissions, need uh, to uh, ah, audit your workflows, see what's happening there. And the thing is it's really interesting to see how the tools that work with that are emerging because the first ones are obviously based on the human flow. And we just had the discussion about Jeff, right. We are seeing more and more tools that are uh, realized that wait, an agent is here, okay, but it's not really a person, it's a piece of software. So we need to, to write tools for the software to use, not for a person to use. We are seeing this change right now. I'm curious to see how it's going to affect this operation design for grading permissions for structuring what agents can and cannot do from the coding agent's perspective, not from the coding system perspective.
Speaker B: Yeah, definitely. That will be a big uh, uh thing that's going to be going to keep watching. Right. Thank you for that.
Speaker F: Yeah. One thing you want to add to discussions of agent is uh, there's two things. One is like harness engineering, I think that uh, an extension of context engineering. But the thing that I see is quite interesting is relevant to a workshop that I'm going to be giving is these agent SDKs uh start out with just helping us to write the loop. But now these SDK has been expanded to provide harness engineering capabilities including memory management for context, including co execution, including some of the out of the box tools for doing interactive local file systems. So that's another trend. So you can see that these agent SDK from Lane Change or Anthropic or OpenAI, uh they all move in that direction of adding more capabilities as part of the harness engineering. That's anyone building agents moving forward. That's something to look out and leverage. And there's a whole other trends of managing these agents. Right. So these companies are providing a runtime environment for you to host your agent whether you want to do it or not. That's up a question but that's what they see. We able to build, we able to build it well and then how do we operate and run it at scale?
Speaker D: I think there is an architectural shift happening underneath all of these. No. So for agents to become useful inside enterprises they need to access tools, data, uh APIs, talk to other agents. So all the protocols like MCP or A2A are going to become more and more different. So I think the question would be whether agent protocols become a new enterprise integration layer. So for example today we connect applications to APIs, tomorrow we may increase the have agent discovery capabilities, negotiating tasks, coordinating work on top of those APIs. So I think all of this architectural uh layer is also changing on the way that agents interact with each other.
Speaker B: Anybody else have any thoughts on uh, quickly how all these agentic AI systems are changing the team organizations, team hiring and everything.
Speaker E: One thing that I can say is that not developing with AI assistance these days is unpractical because very expensive. I mean people are more and more qualified. So people that are uh, qualified to use AI, they carry that on their price tag, let's, let's put it that way. So you have to use it anyway. So once you get there, how do you hire people to fit your own AI assisted workflow? I think that's the biggest question because again we go back to the point that we don't have your standards yet. We're experimenting in companies with different companies, different size. It's very different to work on an AI focused company in a bank or a uh, repay supplier that has its own IT department using AI for something. One common point that I've seen is people are focusing on how well a new hire can actually describe uh, or structure uh, the system verbally. Because we go, but I don't like this. But we go back to prompt engineering. Right. You have to have a full understanding of what you're doing because most people are not really reading uh, the code anymore. I mean it generates all this boilerplate and it's very hard to do a code review these days the way you're supposed to do a few years back. So with the lack of a process or tools to deal with that, people are sticking to what they know or they perceive which is valuable at that time. And the one common point is that new hires that can actually specify the system and get something useful out of AI, something that ah, works within their workflow and then you have the variants. Okay. So uh, if I use a workflow that's based on, I don't know, I have one uh, API model or I have my own workflow pipeline, uh, and then have the specifics. But the common point is that we are kind of skipping the junior level and going to the architectural design level. Because for a new hire to be able to really specify a system they need a certain baggage of knowledge, you know. So guys of generate the problem how you're going to train your new hires if you are not really hiring uh, junior people. Which is part of the mess we're seeing these days. But it's a big discussion.
Speaker F: I do see one thing that's changing as part of the interviewing process where companies are now asking candidate to demonstrate their competency in terms of using AI tools. So it's one of the new models. Not all companies do this but I wouldn't be surprised if more and more company will incorporate this in the future.
Speaker D: Yeah, for example in my experience right now we stop hiring junior developers. I mean junior developers are no longer needed because uh, also the front end, front end, we stop hiring front end developers. Front end is no longer needed. So what we're doing right now with the developer that we're hiring is that as he said, we look for people who knows how to use AI tools who know how to use to implement all the AI into the workflow and the developers are becoming more orchestrators, agents or just orchestrator rather than see the code and the daily things that are happening. So for example in our case we develop a lot of agents for our platform to do a lot of things. So the developers, what they do is to make sure that those agents work fine, that we don't have any, I don't know, data, uh, lake or that we control the expenses of the, the, of the coding, we control all these things that we can control. But the developer, the, the code, the source code, all that stuff is completely out of the of our equation right now. And I think that's where the industry is moving is that it's not that developers are going to disappear is that the developer role is going to change mainly it's going to become an orchestrator of all those agents and make sure that the agents work, they supposed to work.
Speaker B: Thanks Daniel. Also here and I just want to kind of quickly cover the self learning and self improving agents that you mentioned uh, in the offering conversation. So can you share any thoughts on that? Har, that sounds like a very fascinating topic.
Speaker F: Yeah, obviously this, this topic is very interesting and new at the same time. I think it's a progression right? We, we know how to build agents but how do we fine tuning, how do we make it better, how to make it improve over time? There's a lot of research out there in terms of like techniques on how to build these agents that they can self improve over time. The startups are uh, building products where you can give them agents and they will help fine tune your agents. So it's a very natural trends. I ah think we're still in very early days of that but the idea is, it's pretty fascinating to me because if at the end of the day if you're writing an agent most of your logic is going to be in the prompts in the system prompts. Other um, than that you have tools and whatnot but the core kind of logic of your agents is in the prompt. And now we learn like you know, doing evaluations and tracing, becoming very very valuable as part of the agent and how to iterate. So I think these bits are coming together of one day we just got to be expressing what we want and then the system can help fine tuning it and make the agent better over time. But we still in early days. But it's a very exciting space to watch. Yeah,
Speaker B: thanks. Also you shared that link the article from Anthropic. That is very good actually we will include as part of the references. So it goes from uh, it talks about the transformation from like person computer and then uh, the uh, AI solution to like agent working with workers automated basically build AI building itself. Right. So very interesting stuff.
Speaker C: Brought up the point about self improving for agents. And I think the term is like a little bit like ambiguous. Like what does self improving means? Right. On the one hand you can just say, okay, you can tell the agent to never use a rust for example. Right. And it remembers to never use rust in the future. Is that considered self improving or does it involve more of like architectural change? Uh, yeah. What does self improving usually mean in um, clients and from agents?
Speaker F: Yeah, good question. One example. If you think about Hermes as a personal assistant that has that capability to improve over time once it's know you better, it knows how you work and stuff like that. It's one form of self improvement. But that's not the only form. But that's an example of how just it's like a human once they working with you more and more they know what you want, they want the way the style. So therefore they can be a lot more effective at helping you your tasks or what you want to do.
Speaker B: Right here it could include agents capable enough to build and train models themselves with little or no human involvement. Right. So it's the whole life cycle, the whole process is controlled by agents.
Speaker F: That's another form of self improvement. So there's many forms.
Speaker D: Something I would add here is that larger context windows don't eliminate the context problem. So I think agents still need to decide what information is relevant, what can be this curve, what should persist as memory, what information is authority. So I think context engineering is becoming almost like a uh, memory management for this AI system. So giving an agent more context is not necessarily better. If that context is noisy or is stable is contradictory.
Speaker B: Okay, yeah, let's kind of switch gears slightly and uh, Arthur, you mentioned about you are working on multi agents systems. I mean definitely. That's another very interesting area. So whatever you can share with the audience, you know, can you tell us, you know um, how these multi agent systems work? What are some good use cases?
Speaker E: Actually that's uh, a bit of um, sensible topic because when uh, you think about multi agent systems uh, as a topic, it's been there for a few decades. Already right before LLM agents. So people from the academic community are seeing whatever comes with uh, LLM agents and after Transformers and is related to the third multi agent systems, kind of trying to reinvent things that were there in the past. So my research area is on um, classical multi agent systems. So before LLM, uh, um, which doesn't mean that you can't incorporate or use LLM systems as part of it. But these uh, days I've been working on belief revision. So this is a big part of solving two problems actually in the AI landscape. 1 Related to hallucination and the second one related to information accuracy uh, and provenance after being processed by the agent. It's kind of between computer science and philosophy, a lot of formal logic. But yeah, it's really interesting. One thing that we've been working with I think beginning of this year, end of last year, formal logic based profile definition that could be used. It's kind of similar, similar that we can use with the concepts around JET we're discussing before. So have a structured input, a structure output and how we can make the AI not the agent stuff. Uh, I wish to refrain from using agent but actually working from here to there.
Speaker B: Thanks Arthur. A quick one, back to you Hien. You mentioned about spec driven development and software factories. And uh, I've been seeing this in a couple of companies. They're doing this, they're creating dedicated teams to kind of create the software factory model. Can you share more about that? You know spec driven development, how it works, especially for those who may be new to it and uh, what's happening in this space,
Speaker F: this probably started maybe four or five years ago and in terms of like with the coding agent now is you know, pretty powerful. But as you build a project typically you want to have a plan in advance in terms of what you want, the capabilities and whatnot and not jump to the coding agent immediately to do both. So it is kind of borrowing what we know, it's a good practice to do, but now codified it in a form for the coding agent that you work with. So if you build a brand new like a greenfield project for example, you build a spec of what is it about, what are the kind of capabilities that you want? And then you kind of have a range doming session with your coding agent to provide the context that it needs before you actually go and build it. I think there's a lot of good practices there that people have built around this and I think most people recognize that's a good practice. I don't hear Much more about it now because as people are actually practicing it and the AI factory is another. That's probably in the last few months of uh, really how to infuse AI into the various steps of your software development life cycle. The typical use case is hey, somebody raised a bug. How do you get AI to take that from beginning to end of actually take the pr, do an analysis, trying to uh, recreate a problem, come up with a fix, submit a PR and then you have go through the whole, the rest of the steps of review and testing and so on. That is uh, uh, as we all have too many things to do so many uh, tasks and I think if we can help automate that, that will accelerate another dimension to accelerating the software club development lifecycle.
Speaker B: The AI agents fix the bugs, you know, they created in the first place.
Speaker C: I want to ask this question like, about like multi agent system. When I say use like multi agent system, which, which pattern usually go for? So, so for example like is this like one orchestrator is a spin up, like multiple workers? Or like do you have like uh, like different agents working in parallel? And then how do you, and if that's the case and how do you deal with communications like across different agents working on the same kind of like complex task?
Speaker E: Basically the formal definition goes back to agents working together in a social capacity in terms of their interacting with each other to solve a common problem. And in that sense they can have a common goal, shared goals, uh, distributed goals and they can use orchestrators, they can use different architectural patterns. So for multi agent systems per se, the definition is very broad in terms of how they can appear. But in practice what we see is mainly related to orchestration, orchestration communication. So since they have to work together and they have to do that in a social manner, which means communicating. So FIPA standards and things like that, everything kind of revolves around orchestration communication. So I have different protocols for communication, different architectures for orchestration. A lot of work on um, just these two.
Speaker C: Yeah. So I'm very interested in like agent to agent communications. Right. So actually recently I use multi agent a lot. So I have some complex tasks. I would like get a bunch of agents to work together. So I actually built this protocol to let them like okay, when you start working on a task, it's a part of big system. So like as an agent working on the same task, so we can message them to make sure that you guys go like you can coordinate and like don't overlap. So what happened after that is that like my task didn't get done at all. I keep getting a lot of messages and it turned out one agent started forwarding tasks to the other agent and then that agent forward to another agent and then none of them did the work. End up spending like half an hour just like checking, okay, where does this go? We end up doing it and nobody did it. So I feel like communication is one issue, one thing. But like for agent to realize, okay, this task is relevant in my lane and it should do it versus okay, I just want to farm it out to a bunch of other agents. So you needed to be like to be aware of what other agents are doing and the scope of that. How do you manage that?
Speaker E: You try. You don't do it. You try to do that. Because thing is depending on how you're tackling problem. For example, one of the simplest ways of doing that is using a blackboard kind of messaging to see what everyone's doing, what. So they're just posting back. And I have uh, the shared, let's say context for that. But what if you have two multi agent systems talking to each other? The problem starts that you have different IDs, you can have different ID schemas for agents and you have different vocabulary, you have different definitions for the tasks. You don't have a shared states, you don't have shared states, period. And then you have research, you have a lot of things. So for the past decades when you see all the work that was produced, they try to tackle specific problems in specific setups. So for example, how do you tackle the problem of one multi agent? So have shared environment, share everything trying to uh, you have to delegate a task that you can do for some reason, okay, so an agent started doing something and for some reason was unable to keep doing that. Okay, you have to find an agent that's willing to do that. Just for that you need a setup. Are agents benevolent? Uh, do you have to put a premium on that? Uh, or let's take the happy path. All of them are benevolent, are trying to help. Okay, so here comes the big question. How do you define the equivalence of a task to achieving a goal? So I want you uh, to get from point A to point B. Okay, so getting there by car or getting there there by bus, it should be the same or not. They have time restriction. One thing that you'd find when you go deep and deep and deeper on that is that if you don't have a very specific shared setup, a very specific shared conditions, the difficult of a problem increases exponentially. You know. So one of the problems of the multi agent systems area that we have, and we know that we have, is that we have a lack of standards. Okay, so you have standards for communicating, for example, have messaging standards. We do have some protocols in place, but you don't have uh, a standard architecture or standard state description or standard environment description for that sense. That's part of the problem. You have two multi agent systems with two different setups, two different structures, data structures, descriptions, whatever. And then you have to start tackling the particular. So how do you do that? Uh, back to your problem, how do you solve that? You have to start by compartmentalizing your problem. Like okay, this is my box. Okay. So I'm trying to solve this problem in this very specific box with this very specific conditions and setup and descriptions and good luck. From here on as long as you have an um, alien agent coming up or a ah, different multi agent system. So you have different clocks. So start over again. That's the usual path. Not sure if actually answers your question, but it is what it is.
Speaker C: No, it's a very hard problem. I'm wondering if any of you from here and Daniel, have you used the box orchestrations ability of different models? First of all, for in clot or in codex, you can ask an agent, okay, spin up my workers to do this task to make things faster. Have you ever used them and do you have any impressions on them?
Speaker F: I haven't done um, that extensively maybe Daniel.
Speaker D: No, what I do is I work with them like an ecosystem of specialized agents. So for example, you could have one agent which is responsible for coding. We can have another which is responsible for testing, we can have another which is responsible for security, another for deployment. And perhaps you can have a supervisor agent coordinating all of them. So at that point all of the challenges start looking like a familiar to a distributed systems engineering. So you have an identity, you have permissions, you have observability, you have orchestration, you have failure recovery and that uh, communication between components between all those agents. So that's why I think agent engineering will increase, becoming more like a system engineering discipline rather than just uh, prompt engineering or just treat those 80s individually.
Speaker C: Yeah, so when you mention things that maybe think about like the sandbox system I did see from the trend from like this year and last year, it's like a lot of sandbox companies, right? Like building sandbox for like agents. So do you own like using like sandboxes for Asians for example? Each agent had to work in an isolated work tree or something like that. Or it's like dependent on the task?
Speaker D: Yes, for me mainly depending on the task.
Speaker C: So something I've been wondering about. So we have like Compute M architecture right? So aws, uh, we have gcp, we have like Azure and usually a company is a shop right now. Hey um, I'm a GCP shop. So one of my things is run on like Google Cloud or something like that. I wonder if will the same thing happens to AI because right now a lot of companies have subscriptions or have contract with multiple providers. Do you see a future where like people will just stay with one providers? Maybe Open Air can help owners this model or do you think that people would have a multi provider system?
Speaker D: Yes, in my opinion I think it's going to happen the same that is happening with cloud computing. I mean you are not tied to just one cloud provider. You work with a multi cloud providers. So with AI is going to happen the same. You are going to be tied to multi AI providers working on different things of your workflow or different tasks. But I don't see just being attached to only one provider. Like you said, for example I work with AWS but I have multi cloud providers working on different states of my workflow with GCP and Azure. So with AI I think it's going to happen the same. You can say okay, I prefer anthropic but in the end we'll end up working with OpenAI stuff with Groq, with any other provider. So that's going to happen.
Speaker B: Thank you. Uh, yeah, let's. I would like to touch um a couple more topics and we can wrap up. You know it's been a great discussion but I still have a couple more topics. I want to make sure that we share our thoughts with the audience. The first one is the evaluation frameworks. So definitely this has been a uh, big topic uh this year. I even heard in one of the conference presentations that 2026 is the year of AI evaluations. So you know there are a lot of different ways of doing this. There uh, are harnesses and, and uh, the LLMs being used as judges. LLM as judges. Right. So how do you measure AI effectiveness? AI agentic effectiveness in general. Like what kind of metrics do you see in your own experience? Like how do we make sure that uh, the end to end process is being you know has you know checks and balances. Maybe he and start us off on that. Like what are you seeing in terms of agentic evaluation? You know uh, space.
Speaker F: This is a very important topic for companies that are bringing their AI agents to Production particularly the one that for their customers, not so much about internal ones. If you look at example from DoorDash, they build an AI systems for ordering food. You have companies, other companies that billing for customer support agents for external users. And we know all, you know, we all know that LMS are uh, not deterministic and building agents, they can, they can take the wrong turn, they can make the wrong tool calls and stuff like that. So evaluations is kind of a way to help us to make sure that the agents do the right thing for the right tasks. At ah, the first, at the highest level is that and without evaluation how would you know that it's gone rogue or do something that's unexpected, not according to your company's policies and all that stuff like that. And there's like methodology behind that. And we have going to have, by the way we're going to have a sessions that at QCon SF, uh, one of the experts on evaluation is going to be there. And companies not only have to do this but they have to incorporate this as one of their core steps in building AI agents. And a lot of companies ending up building something called evaluation harness which help drive the whole evaluation flow from getting the data, running the evaluation or tests as we all know tests. Right. And then assessing the quality at the end. And that huh helped drive the decision whether it's a go or no go for release. So anytime that you're building serious customer facing agent, you have to incorporate and spend effort and time and effort of doing evaluation. Right. If you build an internal agent you're probably not going to spend a whole lot time on it. But if you building a real serious agent for your company then definitely a must. And you see many examples across companies from LinkedIn, DoorDash, Ubers and such and many cases study on that.
Speaker B: Thanks. Yeah, definitely I think this area is going to see a lot more. It all comes together. Right. So any other analysts, any other thoughts from anybody else? How do you kind of evaluate your own uh, AI um systems?
Speaker D: Yes, I think I completely agree with the idea that this year is becoming the year of evaluations for models and agents. So once an agent can perform an action, evaluating only the model is not enough. So you have to evaluate uh, the whole system. You have to evaluate the model, the tool, the context, the agent loop, the environment. So here is where the metrics changed as well. So it's not only whether the answer was correct, so it's whether the agent completed the task, how much cost, how long it took, how many interventions were Necessary and uh, whether it did something unsafe along the way. So I think all these evaluations were just starting to see how are ah, we going to evaluate all this workflow regarding agents and regarding LLMs and regarding the interactions that they do in our job.
Speaker B: Yeah, good point. All those metrics are more relevant now, right? With the overall. Yeah. One more topic, you know, I definitely want to um, highlight in this discussion is the physical AI. Daniel, you've been looking into this for a while. We talked about this last year as well. How do we bring the intelligence into the real world? Right. So if AI agent is a software system, how do we bring that into the hardware? Whether they're like robots or, or what now they're being called collaborative robots, cobots. Right. And then auto even like self driving cars. Daniel, maybe you can start us off on this. Like what do you see here, what's happening and what we should be uh, kind of paying attention to.
Speaker D: Basically it's saying how do we bring AI into the physical world? So over the last few years we have spent the time putting intelligence into the software. Now we're starting to connect that intelligence to machines that can perceive and act in a physical world. So what you said, we are starting to see robotics, we're starting to see autonomous vehicles, drones and um, industrial systems are now part of this transition. What I think makes physical AI fundamentally different from software agents is that mistakes have physical consequences. The physical AI. So one thing is that in software an agent can make a mistake generating the wrong file, but a uh, robot can move their own actuator. So it's going to be a whole different level of mistake. So the reliability simulation, safety verification evaluations are going to become more and more important, uh, in the physical world. So based on that, all the foundation models for robots that are happening right now, another important shift, uh, is that robotics are starting to benefit from more of the same foundational model approach that transform the language and vision. So instead of programming every individual behavior manually, we're increasingly seeing models that combine vision, language, perception, actions. So is that the road can potentially understand instructions like uh, I don't know, pick up that object and um, put that on the table and translate that into a high level intent of a physical work. So as you mentioned, there are a lot of projects regarding the physical model like Nvidia, like Google Robots, where all these multimodal models are taking language and visual information and producing action for ros. So that connection between language, perception, actions is what makes the space of robotics and physically ah, more interested. So we're going to start seeing things like, uh, simulation for digital twins, robots. All the things that are going to happen for open robotics, all the things that are going to happen from AA agents to physical agents. All of this is just going to start moving faster and faster over the next year. Because right now the AI is here. Now we have to put it in our physical world.
Speaker B: Yeah, that's going to be definitely one of the applications that's going to see a lot more attention, a lot more developments. Um, right. Uh, whether it's manufacturing or any other physical industrial use cases. Any other thoughts? Um, uh, robots and vision language models. How do you see them kind of working?
Speaker C: Yeah, I think it's very, very exciting. Definitely a lot of people say that it's the next frontier to get the models into the real world. And we see a lot of progress. And I think the only challenge is that there are several challenges. So people keep saying that robotics is like cars, right? Like lm, um to say, okay, we know we already have a lot of robots in the real world and we have way more everywhere. We have Dukes. I'm not sure Zoom is out. So I have never driven one, but I see the advertising for it. Uh, so we see a lot of robots in the real world. So people keep telling me that, oh, we know how to train massive LLMs, multimodal stuff. We know how to train robot like survey core. We should know how to train robots to work with real world. But I don't think it's quite the same. So one big difference between robotics models and LLMs M it's that the challenge of backtracking, right. For LLMs, um, when you train thing on a virtual like on the computer, if it makes a mistake you can run back, right? You can reset the state and continue. So you can do a lot of like explorations and pick the best state. You can't really do that with robotics, right? Like you can't get a robot. Okay, there's a squat now this there. The robot fell false and it breaks. You can't just run it back and get the same robot as before. So it makes exploration extremely, extremely hard. And to set another challenge in data. And so it was been looking into a lot of like, uh, robotic setups. So a lot of them have raised a lot of money. So uh, my favorite these companies raised like what, like billion dollars. I don't remember off the top of my head. I think like Yang Le Kun. And so company, we have a bunch of like Gerald like company and general is physical intelligence. Sunday Skilled AI, which, um, I'm, like, very excited about. If you look into it. I was looking into it. And the main difference is the approach on to data. So I think that people are still betting on, like, what kind of data would help robot models to scale. Something tricky with AI models is that, like, sometime when it's small, they don't quite work. A model only work really well when they're big enough, right? For a long time without the language, models, like, they generate like trash and they suddenly boom. Like, you have GD3 and GD4 and it's like suddenly everywhere. So a lot of the bet is like, okay, the models are not working well yet. Is this because the approach is wrong or is it because you just do not have enough data yet? So people had to make a bet on, like, will this guy have, like, architecture with this training algorithm with this data is the right approach to scale? Uh, so we see a lot of this company making a big bet. So there's some companies that, okay, we need real robotics data. So in China, we have, like, a Chinese government is investing a lot of money into, like, have like robot trading center, right? We have, like, humans operating like real robots and just collect a lot of this data. And it's extremely expensive. So a lot of people were like, okay, we do, like, need to scap data. Maybe you can just go with some kind of video. Like, if human can learn from watching video, then robots do it, right? So those are collecting a lot of this in videos. We have, like, people paying people in different countries, like, just like wearing, like, the camera on the head and collect videos. So it's very interesting. But since there are a lot of companies going to middle ground, like physical, uh, intelligence, like, I think it's generalist. Like, on Sundays they have some kinds of globes. So they were like, okay, like, videos are not good enough for, like, haptic information, right? Because a lot of time you can't judge from a video, like, how much force it should put into, like squeezing an egg or like a strawberry. So you need to really take up information so that people have some gloves, like, people have wear them and like, charge manipulate objects and get data to train the robots. So it's kind of like the middle way because you don't have the full robots. It's a lot cheaper. But it's also not just videos. But anyway, I ramble on. I, uh, think it's a very exciting space.
Speaker B: Thanks. Ship. Yeah, definitely. We are kind of running shorter, uh, on the time remaining in this meeting. So I know there are a lot of topics we have talked about the topics that at least that we think are definitely uh, important to be aware of. But you know there are other topics that we can probably, we don't, we won't have time today. But I just want to quickly mention you can follow up offline uh, as separate podcasts or articles. So there's AI governance platforms that are coming up and then there is AI security and regulations is another important topic. A observability is an evolving topic as well and uh, definitely the AI sre and AI ops are also getting a lot of attention. So do you see anything else happening at very high level before we wrap up further? Do you have any comments?
Speaker E: Yeah, I think uh, we could relating multi agent systems and AI engineering because uh, it's not as hopeless that might have seen when I was going through cheap, you know. But uh, there's a lot of things that we could bind uh, together and bring together to correlate to how we solve problems that we have using work that has already been developed. So that would be a great topic.
Speaker B: We can start wrapping up, you know. So this is kind of my favorite part of the podcast to request each panelist to make a prediction for next year and we'll take a checkpoint next year and see how these predictions have kind of been realized. So yeah, we can uh, start off with uh, Chip. What do you see? What's your prediction? If there is one thing you like to see happen or you think is going to happen in the AI space, what would it be?
Speaker C: Really want to see an US robot? Companies that have robots ready to be bought. I feel like, I know it's just like very painful. I feel like in China you can just buy a robot in the US you just cannot get from an American company yet. So if you don't want to see the shipping, I feel like you've been waiting for OpenAI and, and traffic IPOs for a while. I think they delay. Very curious to see like how that impacts the market. Right. Because like if the IPOs go well, the M market will go well. But if IPOs don't go well, what implication would it have on the funding and like the AI market? I would expect to see a lot of like cheaper and faster models like the type safe style of model. Pretty interesting.
Speaker B: Thank you. How uh, about you Daniel? What's your prediction?
Speaker D: I think uh, we're going to start seeing limitations on AI. So for example the phrase is going to be the energy. So all the AI infrastructure is becoming extremely power hungry. So I Think the access to electricity capacity, cooling, data center locations that is going to become a biggest constraint for the growth of AI. So we may start seeing uh, energy crisis for AI. The second prediction that I would like to share is it's like a futuristic prediction but I think is what will happen that all this AI infrastructure is, is going to start moving beyond the, the traditional data center and is going to move into the space now. So I think uh, putting AI into the space is going to be more efficiently. Probably it's going to take a little longer but it's going to be more efficient than keeping the AI hit in the, in the earth. And finally I think the AI competition between United States and China is able to increase in increasely look like a uh, like a technological cold war. So it's not about who has the best model anymore or who has the best chips. So it has to be with who has the manufacturer, who has the energy, who has the data center, who had the open models, the supply chains and the access to compute. So we are already seeing that semiconductor access and domestic chip productions are becoming strategic issues in that relationship. So I think that for today 2027 some of the biggest AI stories may actually happen outside the model themselves and it's going to be more about the energy, the chips, the infrastructure and on the geopolitics.
Speaker B: And Hien
Speaker F: just have two things. Every AI agent will incorporate some form of the Jeff model into the AI agents and more and more of the communications going to be between agent and agents moving forward as we figured out how to like build, operate and bring all these agents to productions.
Speaker B: And Arthur,
Speaker E: I'll circle back to what I said in the beginning. I think and I expect that the most successful AI deployments that we see will be much narrower than what we see right now in terms that rather than one agent with access to everything, we'll see a lot of things using smaller specialized as an agent for mixed models models for well defined tasks. And uh, the big difference will be in the surrounding system. So you have trusted data, limited permissions, tests, a way to see, and this is not back to observability but a way to see what the agent did and whether it actually helped. Teams can not only do that but can actually measure the cost and quality of completed tasks. You can have ROI for tasks, not just the number of tokens of agents they use it. I think that this will be a great leap. It aligns well with what uh, he said on Jed and what Daniel said on the uh, shifting from away from big models. But I think that's where I put my chips in.
Speaker B: Thanks Arthur. Yeah, I think I'll take a quick one minute to kind of wrap up my comments. Yeah. AI is a significant, uh, technology shift. And uh, as we all know by now, after almost four years, it is not a fad or some temporary blip in on the technology radar. It is here to stay. It is one of the major technology waves in the last, I don't know, 20 plus years. Right. So my advice and request to our audience is we, um, should take advantage of this technology transformation, not uh, be afraid of it. Find ways to use AI tools to automate any workflows in your day to day life, whether it in the office or in our personal life. I mean this is a tool that we can use to find more time to do the important things in our lives. Let the agents and the tools do the routine tasks. So my prediction for next year is, uh, definitely, I agree with all of you, AI will be even a bigger part of the daily lives. And I always say this, uh, as a prediction. Like I would like to see, whenever we kind of don't talk about AI anymore, it has become part of the fabric of our lives. I think then we can say we have really arrived. So I think there'll be more evolution, more developments. But uh, I think it's going to, the niche areas will be settled where AI will be really valuable and, and where the human intelligence is still required. Right. So, so thank you all for your time. We took a little bit longer time but there are so many topics and we can go on forever. But I think we need to wrap this up and we will follow up with any other topics later. So thank you for joining today. Have a great one. Until next time. Thank you. Sa.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.