The Benchmark With No Instructions - ARC-AGI-3 (winning team!)

Speaker A: This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new.

Speaker B: It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50 page restoration block.

Speaker A: Or finally, break down that long article you've had open for weeks. Gemini and Chrome is here for it, ready to make anything online make sense.

Speaker B: There's no place like Chrome.

Speaker A: Check responses set up required compatibility and availability. Various 18 plus.

Speaker C: So good, so good, so good. New summer arrivals are at Nordstrom Rack stores now. Get ready to save big with up to 60% off brands like Rag and

Speaker B: Bone, Levi's, Adidas and Free people. Join the NordicLub to unlock exclusive discounts.

Speaker C: Shop new arrivals first and more.

Speaker B: Plus, buy online and pick up at

Speaker A: your favorite Rack store for free. Great brands, great prices.

Speaker C: That's why you rack.

Speaker A: But we have the benefit of millions of years of evolution, right? So it's almost a bit unfair that we're expecting AI algorithms to be able to do all of that.

Speaker C: Exactly.

Speaker B: I think it boils down to one of the big open questions in the field, like, is language critical to intelligence? I, uh, will say, let's say at least when I'm playing the games myself, and I think that goes for all of us, I'm using language. My name is Jeroen Cotard. I have a background in physics and mathematics and I've been working in the industry for about 15 years, with the last five years specializing more towards AI.

Speaker D: It was immediate to him what the goal was. He recognized the pattern and I showed him the game. Within three seconds, he finished the first level.

Speaker E: Often the agents start thinking that reducing the energy bar to the minimum is the goal, or that stepping 10 times in a region is the goal. Which for a human is kind of clear that it's not the actual goal. Uh, my name is Stefano. Um, I study computer science, machine learning. I did some research in reinforcement learning at epfl.

Speaker A: I guess the million dollar question, though is do you think it's possible, in principle, to do really well on Arcade GI3 and be no closer to AGI?

Speaker C: Uh, yes, I think it is possible.

Speaker F: I don't know. Uh, I guess the main idea is a bunch of, uh, bright people in the room and do good research together.

Speaker B: The Transformers can't plan, but they, they can do a very good job of pretending, essentially, that is, in a sense, indistinguishable.

Speaker D: 36% might be misleading as a number if you don't look behind it. So what it really measures is action efficiency.

Speaker C: My name is Dries Smith. I studied electrical and tronic engineering. I, uh, specialized in reinforcement learning for the past five or six years now. And I've been doing on the research side, ARC AGI 2 and 3 and also some quant work in between. Yes, perhaps that tells something about the benchmark. There's some leakage of human friars into the games.

Speaker D: Hello, I'm Michael. Um, I do masters in data science and I'm super excited about arc. Uh, and, uh, I love solving it. In the example that we saw, we have a maze, right? There's no native primitive in a newborn that says this is a maze.

Speaker C: Okay, so this is one of the official ARC AGI games. This is one of the first games that was released. It's called LS20. And here you can see a 64 by 64 frame, which is typically how the, um, game observations is presented to the agent. So you get an initial frame and each of these pixels can take one of 16 colors. And now you need to figure out what the goal is and, um, even how the player moves. Whether it's a specific player or object that's moving around or some other more abstract, um, game, uh, you need to figure, uh, this out.

Speaker B: So very often our harnesses will break here. So they actually got the idea right. You need to move to the exit with the two things aligned. But then they think they should win here. But actually you need to move up one more move. But it is quite common actually, that they fail to beat the level from this point because they cannot think that they think. No, but that's outside the maze, so you can't move there. And the human would just, whatever, just give it a go and see what happens. But somehow we can't get our harnesses to also do that.

Speaker F: The best recipe we have today to build, uh, intelligent systems is scaling up these, uh, large language models. I think Sam said you should definitely not be trying to train LLMs yourself. Nah, I don't recommend it.

Speaker C: Why not?

Speaker F: I think the easiest is just go work at Anthropic. That's the easiest. I have a sense I cannot prove anything that this is the time things start to get weird. Uh, I think he's right about that. Most fundamentally is I want to understand intelligence, how the brain works and how to build intelligent machines. A big mystery. I, uh, guess when I was young I was excited about both computers and brains. Uh, computers. I feel now we have a clue how it works. And I did computer science studies, so that mystery is kind of gone. But the mystery about brains is very much Real life I have a strong sense that uh, most of society is bottlenecked on cognitive labor and insight and basically intelligence. And so if we have more intelligence that would be a big unlock for many things.

Speaker A: You won the preview competition and that was using um, a different method. And then you found that that didn't work very well when the main competition came out and you moved towards this.

Speaker C: Yes. So for the um, agent uh, preview competition last year I actually tried something completely different. So the goal of that competition was basically to test whether there is obvious solutions that breaks the mold, breaks what they're actually trying to accomplish. And there was, and the solution was basically just to brute force actions. So what that solution did was it basically uh, searches through the large action space. For Arc AGI 3 we have more than 4000 actions which makes this difficult problem to do. But you could theoretically search through all of that uh, or just ah, randomly search through actions. What I did for Stochastic Goose was basically to um, search for a large number of possibilities. But I tried to only search through actions that results in a frame change. So initially it, if you took an action that um, did not do anything in the game, nothing would change. Not even the timer bar, um, that allows you to model that behavior to see, okay, does this action change the timing bar? If it does not, we downvote that action in the future for that given frame. Um, there was also only three games so you have to be selective of how you improve things. You can't pre train because you'll just overfit exactly. What I did was I just used an action model that um, learns which frames are valid for a given transition. And then it was more about the engineering of being able to learn within 100,000 actions because that was kind of the max action limit we could get out in the um, time limit we had. So that seemed to have worked well. It solved, I think it got 18 levels completed out of uh, the three games which was, I think it solved two of the games as well in the time limit that was uh, provided to us. But they then hardened the competition specifically against that. So the new competition, the games are much harder. The timer bar moves even if you use an action, uh, that's valid but doesn't really change anything in the game. And um, more importantly they introduced action efficiency. So this makes it uh, very difficult to brute force. You have to be very um, um, direct with how you explore. And that's where LLMs come in. So uh, you could do, even though it's slower just being able to somehow um, guide the exploration helps a lot. It's like the possible combinations is too many to just brute force and obviously your score goes to zero quite quickly. So they've made it so that if you go just let's say 2x or 3x above the human baseline, you're already close to zero. Um, and yeah, uh, even though it's slow, it just helps. Otherwise it's kind of untractable. For us we tried other methods such as directly predicting um, like more of a transductive method where you just have the input frames as context into a long uh, sequence model and you just predict the actions. But it doesn't generalize well and it doesn't really make intuitive sense a lot because if you play the game, for example the first game year, the maze level, you would intuitively play one or two actions and then you'll think about your path and it'll go to the end position. But if you have to think at every step with the same computational um capability or same budget, then yeah, you might be misrepresenting where you should go at the start, but at the end for straight lines, for example, you don't have to think that much. You can just batch those actions. Um, so that's where the coding agent idea came from. And we also had a lot of two good uh, literature results um, where they scaled using opus uh models. Uh, one was Argentica and the other one was the RGB agent which also showed good results. Given no compute constraints. You could use closed source models and we took that as inspiration.

Speaker A: Yes, you mentioned transductive as well, which is quite interesting because um, you know, roughly speaking, I think of transduction as you're making um, a prediction about this specific test instance. And it's quite an interesting discussion whether or not this is transduction because even though it's chain of thought, it looks like a form of induction in the sense that it's a rationale that could be cross applied in the future. So you could use the memory in the agent, you could do some kind of library transfer and make it inductive. But at the moment, if it's only for the sole purpose of this particular problem, would you call it a transductive method?

Speaker C: Uh, so I would call our Oracle GR2 solution more transductive and this slightly more inductive. Exactly. As you mentioned, you can actually read the reasoning trace and understand when it's understanding the game and making progress and when it's not, um, because it has this reasoning chain of thought which is in English. And you could uh, reasonably Understand it. So uh, yeah, I would actually say it's more inductive. Like previous attempts, uh, as we mentioned was where the agent actually just directly predicts the actions, um, that will be more transductive and that doesn't seem to work at the moment. Uh, obviously there's a lot of ideas to try and I'm sure the community will come up with something interesting to make that work. But for now this seems to be the way for us.

Speaker D: Yeah, I think the action efficiency makes uh, this problem really interesting. Um, because there's so little time to learn and you really, if you want to get the a hundred percent, which is the, which is the grand prize, then you really, you, you have to outcompete a human which really can take a lot of time just to think. And time is not a in question for the solutions. So I think that's, that's really interesting. And the reflections and looking back at the past, uh, at your past experiences makes LLMs very flexible, uh, model, uh, for the solution. Um, yeah and interestingly enough also a lot of, even though there shouldn't be any priors, there's already a lot of the game priors are already included in the LLMs. So for example here we're looking at the maze. Um, and maze is something that every LLM, even the small ones, uh, will know and will recognize from be it images or ASCII grids. So even though uh, all the priors are stripped away, there's still enough game priors that the LLM can lean on and those are not encoded in any reinforcement learning model or like a pure um, goose solution wouldn't have that encoded anywhere in it that a maze is a thing. Uh, and that really helps to direct the reasoning and the actions of the model.

Speaker C: So one of the main constraints was basically two weeks I joined the competition late, uh, two weeks to do it. So there's a lot of things that can be improved. But I think it was a good initial solution. So the main constraint was we only had three games, three public games and three private games to evaluate on. So we had to make sure, to make sure that whatever design is, you don't assume too much about the environments and the games because on a private leaderboard or like the public leaderboard beforehand you could actually see that I wasn't even close to the top because it was super easy to overfit. Um, so the basic idea was um, we can't pre train, uh, we don't have enough environments. Let's try to do test time training like we did for RKGI2. But how do you do that? You can try reinforcement learning, which I have a lot of experience with. But um, it's not that obvious how you do reinforcement learning because every time so you get a new game at test time, you have only real reward you have is a level transition. But once you transition a level you never go back. So it's not like you're trying various uh, uh, routes to try and optimize for passing level. You just have to pass it once. So it didn't really make sense to use pure um, reward based rl. Um, I did a lot of research in the past in curiosity of unsupervised rl and I thought perhaps that's a better approach where we don't have explicit rewards, we're just optimizing for um, curiosity like exploring new areas that the agent hasn't explored before. Um, so I tried a bit of world modeling. So the basic idea is the game's not deterministic. So you can use a world model to perhaps have um, the input frame, then the action and try to predict the next frame. And then if you can't predict it well, you have the policy explore that more and that's the reward you're optimizing. So that was a promising approach. I couldn't get in it working in two weeks. So what I just then assumed is that any frame change is interesting. Uh, you want to explore things where the state changes, you don't want to explore things where the state stays the same same. So that was the basic derivation of the or uh, path. Um, I followed towards getting the action, um, model, uh, implementation for stochastic goose. And then it was down to basic uh, reinforcement learning. Things like how do you get it to learn within 1000 uh, timesteps because you have 100,000 times, but you need to start taking useful uh, actions in like a thousand time steps. So it was like using a replay buffer, hashing the experience so it doesn't go over the memory limit. Um, do some uh, prioritized experience replay and some basic engineering stuff um, to get working for that uh, competition.

Speaker A: Yeah, you mentioned exploration as well. Um, I guess there's a bit of an elephant in the room which is that Chollei is talking about the acquisition and synthesis of abstractions. And when reinforcement learning folks talk about exploration it seems to be in quite a surface superficial way. So in terms of like entropy or things changing. And do you think, is that in any way against this idea that we can acquire deep abstractions about the Domain, yes, definitely.

Speaker C: For Stochastic Goose, uh, it was not at all uh what I guess what Frontra uh at the end wants as a good Solution for Arcade GR3 but I can't speak to what he actually thinks is a good solution. But that was purely to see if I can benchmark maximize this competition uh within two weeks. I think the duck and things uh we are doing on that side is perhaps more uh, in line with what that initial vision is. For example you have this reasoning tokens which has abstract representation of objects which we then manifest. Um, or write down in uh English language textual tokens. Um, so I would say that's more abstract. Identify objects, you try and find out what the mechanics is, dynamics of the game, what the goal is. Um, that might be more abstract. Uh, and it's also kind of like a neural guided search. And also the programs, the Python programs it creates is like executable Python programs to extract and also build some sort a uh, simplified world model and search over that world model using actual algorithms like breakfast search.

Speaker A: There is understanding debt, right, that you're building this really really complex thing and after a while. Have you noticed in Claude code they don't even show you the code anymore. Right. You can expand it but by default a lot of people don't even look at the code. So do you think that you need to be familiar with the deep abstractions in your code base in order to kind of build mental models and evolve it and extend it, you know, do you get lost in no man's land?

Speaker C: Yes. This is a. Yeah. If you. This is an active discussion within our team. M. We um, don't have a discrete answer yet but I think uh, like what we've found is like you need to deep understanding of um, some important parts, some other parts like let's say a web viewer, you can vibe code more easily because that's not really. If it breaks it's fine. But the core logic for let's say the implementation and also evaluation is especially important. We have to make sure it's working. But it is a case that uh, gradually we are understanding less and less of our own uh code base and we're struggling even with reviewing some of the changes is you might use a codecs to help review some of it because it's such a broad change or we need to split it up. Um, but things are moving so fast that you can't just manually. If you just manually write everything you can't keep up with the rest of the team. So uh, yeah I guess this is active discussion. It's difficult for me.

Speaker B: My career has been going on a bit longer and let's say for most of my career there were no coding agents, which means also a bit of an opportunity to still make use of patterns that have been useful in the past. And I think one important one that as a team we are more and more learning to use properly is requirements based engineering. So we will formally write requirements, let's say really following uh, uh, let's say detailed prescripts like numbered requirements and to specify how they are tested. And I mean we might still have the coding agents helping there, but mostly handwritten. We will review that as a team and from there the coding agents do. We can much more confidently hand it to a coding agent to implement than if it's just a viper single prompt.

Speaker A: Yeah, I love all of that. But I suppose there's always been this thing, right that um, in software engineering we have specifications, we have documentation, we have user stories, we have requirements and all these things. And uh, there was waterfall versus agile. For me the difference between waterfall and agile is this admission that there is epistemic uncertainty. Right. So we need, you know, like reality pushes back, we discover contours in reality that we didn't see before and we need to adapt the requirements over time. And my worry is that when, when you kind of crystallize the process. Um, I guess what I'm saying is there's more to tacit knowledge in developer teams than the requirements. There's, there's, there's the generative process that created the requirements in the first place. And when you kind of only write software for the purpose of satisfying the requirements, then the ability to evolve the requirements gets kind of deteriorated.

Speaker B: Yeah. So what is always the case, and that was the case in classical, let's say pre coding agent, uh, development as well as it is now, uh, is that it is not. You write requirements and those requirements are frozen in. Yeah. Carved in stone. And then you write code. It is an interplay. So when you start implementing you might realize, wait a minute, this combination of requirements is not possible. You don't then just implement something that approximates the requirements. You go back and we also kind of uh, tell our coding agents to do this. You go back and say, hey, these requirements don't work, they need to be adapted and you make a conscious choice there. So we will also very often kind of go back with a coding agent. Hey, can you very carefully check that every requirement is satisfied? In fact we have it write specific text per requirement like this is how it is being satisfied in the code. Uh, we try to also really write the tests around the requirements rather than just having kind of random tests for every line of code. And then you get kind of that classical interplay that you also have in classical engineering you can do with your coding agents.

Speaker A: This is a kind of um, AI human co creativity. So humans um, have the taste, they have the intention, they have something that the coding agents don't have because you know the bull case. Wouldn't it be amazing if you could stick the requirements in and the agents themselves would kind of understand. Oh, I see what they meant. And they could evolve the requirements.

Speaker C: So uh, I think in the short term, I can't say for the long term at least what we're trying to do is find the areas where we can see the model still failing, like how high level conceptual ideas about the problem. And then we try to make sure when they say we're reviewing code or creating code that we ask specifically around that area and make sure it's correct and understands as well. Um, but yeah, I guess it's always going to be a game of cat and mouse where we're going to perhaps uh, hand over more to the AI assistants. Uh, but in some areas it might just totally fail and we need to take responsibility there, make sure we understand what it's doing and it understands what we want. Um, but yeah, longer term, I guess, uh, if you are AGI pulled or super intelligent, I guess it's up for debate.

Speaker D: So it's certainly accelerating us in doing experiments because something that would just sit in your scratch pad, you can just paste it in a prompt and see if it sticks. So you can definitely experiment overnight with a new idea. Um, I think really the added value that I see of the lab right now is uh, and what the AIs are missing, what the coding agents are missing right now is like literally the attention to detail. So this task is very long. Like if you have a long context, the execution of one game might take an hour, um, just for it to solve one level and just seeing. And if you could in principle point auto research at this and say hey, just improve the score, right? But that doesn't work because it doesn't really, it cannot really. We have tried that and that has not really led to improvements because it doesn't really understand where the agent fails. Um, it's not simply as Andre Karpathy's other research challenges, where you just minimize a loss and there's just parameters. Um, you need more insight to this task. And I think that's where a lot of improvements came from. Not from, it came from reading deep into the program, into the logs and understanding what the issues are and improving those one by one. Rather than just like vibe coding another 2 million lines of code with coding

Speaker E: agents trying to optimize this task. We observe kind of similar failure mode, uh, that what we observe when they play games. Like for instance if you give them an auto research task they might over focus on details and try kind of not see the big picture and not kind of zoom out um, and see things from Fireball and they might, I don't know, start optimizing some hyper parameters like at 0.01% improvement while there is some other big improvement that could be done which is kind of in another abstraction level. And we see very similar failure mod in games where um, if the agent at the beginning of the gameplay, uh, they get kind of locked in on the wrong hypothesis, it's extremely hard that they escape that they kind of convince themselves that that's the right path and the right hypothesis to go and uh, they're not able to escape left. So it's quite interesting that um, yeah, across models like smaller and frontier one, we observe kind of similar failure mode of them kind of not being able to jump between an abstraction level and not being able to um, zoom in in the details and then seeing the higher level picture. Uh, at the same time I just

Speaker A: wondered like how do you guys think about the abstraction mountain? Because for Chollet it is bottom up. So you start with the core knowledge, you know like agents, spatial knowledge, stuff like that and you kind of synthesize upwards. Whereas LLMs are kind of interesting because they learn this, these fraction entangled representations that are quite high level but also quite generalizing and then you can stitch them together and you can repurpose them and canalize them in different contexts. But it's always a little bit lossy. So if you're doing this library learning and transfer, you always have this issue of like well yeah, there's this abstraction over here and it kind of works, but it's not quite what I want but I can still use it anyway. I mean is this, is this kind of reasoning something that should be first class or do you think it could just be implicit in the harness if we had a memory system?

Speaker B: So you were just describing how LLMs will have uh, indeed maybe let's say the core knowledge priors as something somewhat vague but not fully testable but still usable. Um, the thing is, if you replace LLM by human there. I think it's still true. We also don't have formal core knowledge priors that are let's say fully exact and that we apply to the world and describe the world fully exactly. We have kind of vague trained over our life understandings that evolve over time. So at least if you want human intelligence, that does seem to be the paradigm you want there. And I think the LLMs get kind of the bonus that they also can reason very well and much more implicitly than the human scanning code. So if they want to get exact, they can. But I think if you want to understand the world certainly beyond these, let's say simple games, you will need more probably that, yeah, vibe understanding. Basically.

Speaker A: I'm guilty of always like you know, suggesting what Charlie thinks. I don't really know what he thinks, but you know I've got a pretty good simulation of Charlie in my mind. But uh, I, I think he's influenced by this kind of nativism, um, psychology type thing. And, and he thinks that a lot of the reasoning we do do is almost um, Platonistic, you know, that, that you know, somehow the laws of nature impute these primitives into our mind and we compose these primitive primitives together for certain classes of problems. So we can DO Abstract System 2 Reasoning. And for those types of problem we do compose these things together. But you're absolutely right. There's so many things in the world are actually really complicated, right? You know, like navigating relationships or even you know, pathfinding in a complex environment, you know, on the tube network or something like that. So we do a little bit of both, but there's at least a pocket of pure reasoning. I think that's what Charlie thinks.

Speaker C: Would you consider uh, LLMs pre trained on the Internet? And then if you apply it to a new pattern using like in context learning, is that a form of, okay, let's say with reasoning as well. Would that not be a form of um, like perhaps skill acquisition on the fly because it learns. Obviously there's a lot of core knowledge as well, but there's also a different part where it has to adapt on the fly. So you can give some example that might not be on the Internet and might adapt. Sometimes it fails. It is still a distribution, but uh, at least you get some adaptability. And then for example when you started prompting it uh, like uh, chain of thought prompting, it started improving because it had more time to reason and adapt to what you're asking. And then if you train it using reasoning, you can actually do way More because now it has more time to actually reason and figure out what you're actually asking and form new extractions, um, for solving a specific problem case as opposed to just um, regurgitating what it's seen on the Internet.

Speaker A: LLMs trained to do reasoning. They are intelligent because intelligence is just adaptation. So you know, I know this now and I need to combine what I know several steps, you know, to solve a particular task. So in the partial knowledge regime where we have a verifiable function and we can do hill climbing, um, but what we see though is that they combine together fractured and tangled representation. So they get the right answer but for the wrong reasons. So they create a spaghetti monster. But what we do is we take um, a more valid path and we can acquire and reuse abstractions. The models themselves, they don't seem to work bottom up. They work, um, they have an understanding, they have representations, but they're very fractured and entangled. And that's still useful, but that's a kind of statistical low level intelligence. And I think you could in principle still do abstraction from the fractured entangled representation. But we're kind of in statistical land. We're not. It's performance, not competence.

Speaker C: I don't have strong views on this, but I have a feeling that we are doing the same with our minds. Like if you look at the evolutionary process that derived us, we're kind of um, where we are at at the moment is also we've been evolved to work well with spatial information with like that's one of the core priors in our brain and some level of reasoning. Um, that might also be why intuitively understanding quantum physics is so difficult, is because we kind of overfit to our world and how we've been evolved. Um, so yeah, I'm still not sure whether the human brain is also just its core priors and pure reasoning. There is some guided search, uh, to find answers based on what we found through evolutionary process to actually work well, um, to actually survive in the world and um, propagate.

Speaker A: It's the million dollar question, is it because our brains think symbolically, but our brains are actually just a bunch of neurons? So is it a post hoc confabulation or is it actually like a causal basis function of our cognition? And if it is just kind of confabulation and it's not the basis, then maybe AI doesn't need it to be the basis either. But Shollet, I mean, I don't know if you guys want to talk about what Sholay is doing, but he's building this approach where we actually um, acquire and synthesize program graphs using neural networks as guidance.

Speaker B: It's a tricky one because for me, coming mainly, let's say from a more classical machine learning, Bayesian perspective, it feels like this should be the right way to go. And certainly when I learned about these things and kind of very broadly also kind of sweeping stuff like Dreamcoder into that uh, regime, although it's not quite the same thing is it definitely feels like it's more natural to think of it in that way and perhaps more natural to the way we reason. At the same time it is not competitive at this point with LLM based approaches like um, who knows, who knows what happens in this competition. But we don't see those approaches necessarily being the best on Arc AGI 1 and Arc AGI 2 and we really don't see how we could use them on arc AGI3. That could also simply be because they haven't had the same attention, the same resources put into them. But yeah, there is that mismatch. Why don't they work if they match our intelligence so well?

Speaker A: And I would even um, I'm trying not to use the word symbolic AI in general because I think the more abstract way to think about this is constraints. And Chollei's been using the language of constraints. And we wrote an article about creativity. And I think that creation, whether it's intelligence so in a known domain or whether it's creativity in a transformative unknown domain um, is all about just respecting constraints and acquiring constraints of a problem domain. And this is why harnesses and tools and code work so well. Because you're increasing, imbuing domain constraints um, into the learning and acquisition environment. And so we find ourselves in this kind of halfway regime where we have a constrained coherent execution environment. But it's only really steered and legible in the sense of these functional descriptions that we're talking about. So rather than having some um, low level bottom up competence, we have a kind of intermediate competence which is as described in a similar way to the Blind Men and the Elephant. So from many perspectives, many functional descriptions from many perspectives, perspectives and we're using that as the basis of constraint building and acquisition of uh, knowledge.

Speaker D: What I find really interesting about the core knowledge priors and what you said, like we have uh, these representations and they're very fractured but somehow they work. I think uh, there's two levels of abstraction so um, two levels of intelligence that human has. Like there's the corner prize and then there's what we learn through life. And the games that have been tested with humans, have been tested with humans that were not infants, uh, for our challenge. Um, and there's a lot more that we accumulate over generations through education and so on. And those abstractions are way more powerful. In the example that we saw, we have a maze, right? There's no native primitive, uh, in a newborn that says this is a maze. But this has been over generations of problem solving through humans, uh, very much established concept. And if we can inject this into an LLM, we can skip many levels of abstraction. Um, and I would say that's why when we play these games, some of them for the first time, they're also very difficult for humans because if this is done well and there's nothing like a maze in a game, and it really boils down to the core knowledge priors, then it's very difficult even for a grown up adult to synthesize the correct rules. However, most often, um, and that may be a failure of Ark, uh, a little bit in its design is that these games usually have some sort of, oh, you're shooting a ball or you're solving a maze, you have an enemy, right? And these are like as pure as they're trying to be. They're never going to be pure because they're made by humans. Um, and that's why we can do so well with LLMs, I think, because a lot of these, like maze is the canonical example or uh, I think uh, like those can be really well done with LLMs.

Speaker A: They have these fractured kind of, you know, fractionated understandings. But isn't it interesting that when humans use LLMs, we can make them act as if they understand because we can say, well, um, you can put a framing and in the prompt you can say think about this problem like it's a maze or think about this problem like it's tic, um, tac toe or something like that. And what you're doing in the prompt is you're basically setting constraints on its generation. And then you know, like the more you work on a software engineering project, you know, like it doesn't quite understand. You put more constraints and you update the prompt and it's like it converges and after a while you don't need to repeat yourself, it understands. And so it's almost as if when you get LLMs in the mood, when you make them track this perspective where it respects the constraints, then it understands. But like the million dollar question is how can we just make it understand autonomously? How can we give it a novel domain and do what we do? Which is like saying ah, this is basically a maze. This is how you need to think about this and get it, get it into the understanding.

Speaker E: Good way to look at this is to look at the function um, that they learn during training because like uh, LLM becomes very good at understanding languages and operating on anything that is kind of language related. And then all of the work that we do in the harness is trying to um, put this novel domain into language. So like trying to kind of represent um, the game or whatever domain you're trying to make the LLM act on in a way that is kind of language related so it's closer to its pre training prior and it allows to act on it. If you're not able to do that since the LLM has this kind of prior from pre training it will never uh, it's way harder to make it act on game because it's such an out of distribution uh, domain compared to what it was trained on.

Speaker B: Yeah, I also want to comment a bit. So you mentioned indeed that what you often uh, I think is easiest to do with coding agents, your analogy where you kind of have to say yeah but you need to prompt it a few times before it really gets what you want. Um, but I don't see, let's say having worked a lot also with instructing human teams of software engineers for example, I don't really see a huge difference there. In a sense that you cannot expect to just tell someone in two cents, I want something like this and then get the right thing. That's a failure on your part not to specify what you want. And um, we see indeed with this requirements engineering, requirements based engineering, which takes a lot of thought, we might spend days getting the right requirements but then it is I would say reasonably well capable of doing the right thing to begin with basically. But yeah, it's also our job to specify it, right.

Speaker A: I call it perspective or blindness. So you hire a new person and you build an internal wiki and you specify all of your learnings and you think oh, uh, I've distilled this, I've crystallized it, the new starter will just be up to speed within a couple of days. It doesn't work like that. There's almost like experiences, this weird tacit, almost like a causal graph that has a provenance and it's only when the new starter plays with all of your tools and tries different things and has counterfactual representations and whatnot. So yeah, it's a very Human thing as well. I mean the general thesis is that knowledge is quite path dependent. It's not about where you end up. It's not about these abstract functional descriptions. Sometimes it is if they're so incredibly idealized, like calculus or something, they're the basis of, of evolution on their own. But so much tacit knowledge actually requires the path of how you got there in order for you to continue the path.

Speaker D: Yeah, I think it's really especially an ARC challenge. I think it's really interesting to consider and would be really nice to see what the, when they were testing the game. Like what when the humans failed, how did they fail? Because, um, at first the scoring was based on the second best human attempt for efficiency. And that is very skewed, um, uh, towards a good solution. Right. And then it was changed to a

Speaker C: median I think second best per level as well. So it was almost impossible even for the best human to solve a game.

Speaker D: Yeah. So it was, ah, so it was based on every level. Um, and it was really difficult for a single, even a single human would like educated human would not score 100% on this benchmark. Right. Um, and we don't know much what the failures are. So we have played these games, we have tested them ourselves let's say, and sometimes we don't get them. Uh, so it's not like I struggled with some of them. Eventually I solved them. But uh, sometimes I would spend like on some of them, I would spend an hour to solve them. So you cannot really say this is something that's immediate. Uh, it's also difficult for humans. And I think it's really um, like looking at these counterfactuals, like what do human engineers Ms. Vs software engineers as Jeroen just said. Or like uh, what actually humans fail on these tasks rather than oh, human can solve it if you give it 10 attempts or something.

Speaker A: Really interesting point. Yeah. Because so this, this is the um, the human difficulty calibration. And on RV2 for example, you know, the test was let in if, if 2, any 2 humans could pass the task. Yeah. And this is a bit of an ideological thing because I think Chollei believes in this notion of general intelligence. So in his mind, someone with general intelligence because they have the core knowledge, they could, you know, inside, let's say a certain number of steps in the combinational closure of these core knowledge primitives. Any human with a certain amount of general intelligence could solve any of these problems. The counter to that is that as we've just been describing, maybe human intelligence is Quite high level and fractionated and pattern based and it's quite specialized. So some humans can solve certain tasks because they have a certain perspective, they have certain experiences and whatnot and they can just, they can combine those fractured representations together to come to an answer. And, but maybe they're just in possession of those representations and another human isn't.

Speaker D: I have a friend that uh, is a professional esports player. I showed him one of the games and he completed the first level without spending an extra move that he didn't have to. It was immediate to him what the goal was. He recognized the pattern and I showed him the game within three seconds he finished the first level with sub, uh, super human performance let's say. And okay, he has been playing professionally games for five years, right, but, and he's been trained on this but like exactly as you say. Like there's something we can bootstrap on and we do bootstrap on and it does work. So maybe replicating that with LLMs is enough.

Speaker A: Yeah, I think gaming's a really good example. I mean a good friend of mine is called DDK and he does um, commentary on Counter Strike and stuff like that. And it's when I, when I watch him do commentary it seems super situational. So like in, in Quake 3, you know that there's, you got to time the mega health and then there's the red armor and then there's a, there's a position over here and what you get is like the emergence of these complex situational phenomena in the game. And this is, this doesn't seem anything like what Chalet is talking about, you know, because there's two worlds, right? There's the sort of, the emergent complexity world and then there's the, the sort of reductionist, you know, kind of core knowledge world. I think in this world we can still use intelligence and we can still absolutely like acquire abstractions and descriptions for these high level phenomena. But, but what do we do? Do we analogize them in terms of low level knowledge we already have or are they something new? It seems like a different modality.

Speaker D: No, I don't really have a good answer to that. I just only I think I can base my answer on the, on these examples and I completely agree with you that these, that there are some abstractions like that. But I, I don't know, I don't know.

Speaker A: Like, but you know, like Conway's Game of Life, um, it is still path dependent, right? You can still trace a path from the low level description to the high level phenomena. But the reason it's unintelligible and it's surprising is because the path is a long one and it's computationally irreducible. So you have to perform every single intermediate computation to get there. And it's confusing for us because we can't actually analyze or understand the path between the causal stuff that made this high level phenomena. So we just kind of see it on its own like disconnected. And many phenomena in the world are like that.

Speaker D: Yeah. Some sort of like emergence you're talking about like where just you have to call it emergent because you just can't trace. You're not capable of like the computational graph of that is just not tractable for you to.

Speaker A: That's true. But another feature of emergence is compression. So like with the glider for example, we almost don't need to know how it came about because the new description encompasses, it's a complete description of the new phenomena. So it's almost like it's the start of, of a new piece of knowledge. And, and a lot of uh, our knowledge is like that. It's such a good compression of what went underneath that we can start from it. So like for example, let's see, you like calculus, you know, for example. So calculus is within the closure of calculus or probability theory. We can do a lot of things. We don't need to know how it came about. So, so but, but then is, is that a new layer of knowledge and we don't care how it came about. It's just kind of like saying do we need to start from the bottom? Or in many cases uh, this has come about and we can just use it higher up the abstraction mountain.

Speaker D: Yeah, I think ARC is an example for the case that you can do this at least to some level because we see the frontier LLMs scoring quite well on these. End of the day wouldn't be able to if these core priors would be breaking it. But I think what is really nice about ARK is that there are some uh, games that are so different and are still easy for humans but, but they somehow break this concept. Right. You talk about this closure of like, and I gave the example of maze. Uh but like you can maybe change the core prior concepts and they would still make sense for a human, but they would somehow break adversarially. This like, let's say calculus like maze like representation that helps solve the game. You can like move them around a little bit. Then you suddenly break it and you still have a valid game for a human, but not for the abstracted. LLM intelligence. Uh, and that's what we see also with the, with some of the difficult games. But I think it's really difficult to not move the priors too much so that you can still make it human, solvable and that sometimes Ark unfortunately fails at this.

Speaker A: I want to talk a little bit about RKGI3. So the first two versions of the ARK challenge, they were quite abstract because Chollei was trying to um, idealize intelligence on its own in the most abstract kind of legible way. Our RKGI3 it introduces, in my opinion, and we can talk about this, the concept of agency. So agency in my definition is the ability of an agent to have goals, to plan and realize those goals. And the more ambitious the goals are, and the more you realize the goals, the more agency you have. So there's a kind of low level, no nonsense definition of agency, which is just a thing that can sense and act. So that's like if I'm doing computer programming, that's what an agent is. But the more kind of cognitive science definition of an agent is one that sort of has future pointing control. I have these big goals in the future and I can realize them and that makes me an agent. So I, you know, I, I think that RKGI3 introduces agency not just in terms of realizing goals, but acquiring goals.

Speaker D: Um, yes, agency. And also like acquiring these goals over time through interaction and adapting them. Because not only do, uh, do the levels on their own to be solved require interaction, but all the levels change. So these goals dynamically change and you, so that you, why you cannot uh, learn them, learn them directly. Um, and then on the, on the concept of agency. So I would, I would argue that one of the interesting things that we saw is that what I thought coming into this challenge and started working on this problem is that it would be difficult for the agent to have any sort of idea what it should be doing at all when it tries to solve the first level. But I think by now we got at least one solve on all the public games. At least the first level. May that be by trial and error, or may it be by luck. But still there is some. Somehow LLMs seem to be able to figure out what they need to do. That might be through the different biases that are already encoded for humans that have been building these games. But somehow it is possible to generate these hypotheses. They may not stand for long, they might be kind of random, uh, but there are signs of life that LLMs can already do.

Speaker A: This I would argue, certainly like there's that anthropic biology paper and they kind of, you know, talk about all the capabilities of language models. But one thing that they said language models can do is plan. And planning is required for agency, in my opinion. And we know that language models can't plan because planning is a type of computer science algorithm that is not possible in the, in the substrate of Transformers. So what they do is a, uh, kind of simulated planning.

Speaker D: Right.

Speaker A: So that they can simulate planning up to a certain depth. Do you think that matters?

Speaker B: Maybe. One important thing here is to realize that there's two types of planning that are involved in playing an ARC game. So there's, uh, um, you need to, at some point, when you understand the rules and you understand what you need to do to win, you need to plan a path to get there. And that's kind of more of a traditional game theory planning algorithm. So you might use a search or Monte Carlo algorithm or one of many such things which the model is perfectly capable of because it can write Python code and run the Python code. That's one level of planning. But the other level of planning, when you enter just a level and you see it for the first time, is you need to figure out. How am I going to figure out the rules here? Oh, that looks like it might be a player. Let me try to move in a direction. Ah, okay. Pushing up moves it up. And you need to balance that exploring and exploiting. For example, when do I start trying to win? When do I start trying to, um. Yeah, how long do I start playing around? Basically, that's a whole different level of planning, which you can't encapsulate in Python code. And let's say we see our agents, actually, I think it's fair to say, managing that quite effectively. So in a sense, yes, maybe Transformers can't plan, but they can do a very good job of pretending, essentially, that is, in a sense indistinguishable from what might be formal planning in a computer science sense.

Speaker A: And I hadn't quite incorporated that. You're absolutely right. So even if they intrinsically, they can't explicitly plan, they can call tools, they can write code that can plan. And there's a bit of a Galaxy Brain side discussion there about whether they can effectively explore the space of programs to do planning, or whether it's enough that they can statistically generate programs that do enough planning for the circumstances they're used in. Maybe that's too Galaxy Brain for this, but. But they can basically plan. In essence, that's what you're saying, yeah,

Speaker B: and so what it comes down to, again, whatever is, let's say under the, let's say galaxy brain understanding needed to understand what's going on, they seem to be able to do it to a degree that lets them play these games essentially, uh, whatever it is they're doing and whatever it really means.

Speaker A: Okay, well in which case, where's the gap then? So if they can plan, is it a representation problem? What's the big gap?

Speaker B: So maybe it's first good to discuss what the actual gap is. Because indeed, if we see for example, what the ARC is publicly communicating, which even Chile, for example, they show simply the benchmark where even frontier models are not able to play the games to any significant degree. So they score under 1%, uh, which either means they're not beating any levels or they're being very inefficient about it, doing way too many actions. Um, however, if you look at publications, you can see that score can go up to, it's the best now, 35%, 36% if you give them kind of a, a proper harness.

Speaker E: Yes.

Speaker B: So the point is they need some guidance. You need to tell them more like, okay, you're in a game, you're going to do this, you can write code, but with that guidance, they can actually do pretty well. And that guidance, it's not game specific. It's not like, oh, maybe this is a maze and you need to think about this. No, it still needs to. The model still needs to do that explore versus exploit balance. The model still needs to do all the move planning, but it gets some kind of guidance on the general thinking patterns that might get it there.

Speaker A: What we know is that these harnesses we can, with a little bit of work, make them do anything. It's a little bit like, for example, if I already know how to solve a problem, I can use GPT 3.5 and it will give me the right answer because I'm giving it guidance. So it's a little bit like a heuristic search problem where if you guide the search enough, if you place enough constraints in there, then statistically, like if you sample enough times, it's going to get the right answer.

Speaker B: Yeah. So I think that very much is Chalet's. Look at this. So that's also why they don't the scores are under 1% because they don't allow harnesses for that leaderboard. Because his point is, yeah, you shouldn't need a harness. A human doesn't need a harness. It's a tricky one. I, for Myself don't know if I really fully buy that argument in the sense that the reason a human can do this, uh, without needing a specific harness, I mean that's fine. But most of the people, especially people who are capable of playing these games, have played a lot of games, be them computer games, video games or games in real life. So they've already learned the thinking patterns that we are now encoding through a harness. Much, uh, more directly. I could also imagine if that hypothesis is true, that it works that way, that if you were to take those 1% scoring things, but you do kind of a, uh, reinforcement learning loop specifically to teach them gameplay, but without hard coding any harness, they might actually do a lot better. And does that count? Does that mean they can then play it? Well, I don't know.

Speaker A: But is that the same as arguing that if you went to a deserted island where there was some tribes folk that have never used computers before, do you think they would be able to do Arcade GI3?

Speaker B: I think not immediately. But if you give them 100 of these things and you give them one per day and you reward the people doing well, I would assume at some point they can. And I do think it's fair to say they probably need a lot less training resources in this amount of, I don't know the amount of compute required to learn it than right now a frontier model would use. And even when we say they can reach 36%, that is true, but it costs like a few thousand dollars, um, which is a lot more. Although it's hard to of course want a lot more compute than the human beating these games.

Speaker A: Yes.

Speaker B: Is spending. So yeah, does that really count? As the old models can play games now if takes them so much effort, I would say that is a real gap. And also I'd say being able to do it frontier models of. No, sorry, with small open source models would demonstrate a lot more is indeed what we're doing in the cattle competition.

Speaker D: 36% might be misleading as a number if you don't look behind it. So what it really measures is action efficiency. So uh, correct me if I'm wrong, but it's the ratio of the, of the human baseline divided by the number of actions, uh, the AI has taken, whatever model it is on the level, or a human, whatever the player is, and then squared as well. So this plays really adversarily to anything that's a little bit action inefficient. So 36% actually in this case doesn't mean we solve that approach solves 36% of the games. It solves way more of the games from the training set. At least this number is on uh, but it just solves them inefficiently. So I think this is really important to emphasize that the current frontier models are able to solve what is it, something like half of two thirds of the training games actually till the end, but just not as efficiently.

Speaker A: What is the hardest thing in Arc AGI 3? Is it the goal acquisition or is it just simply the action efficiency from

Speaker D: what we see on the training set? So the testing set, uh, the private set is set to be harder. We don't know anything about it. Nobody outside of their organization has seen it. Uh, so we don't know how hard it is. But at least on the training games we see that the LLMs can acquire the correct goals and pursue them um, somewhat effectively. Um, the question is whether this holds also for the witherheld private test. But um, it seems that goal setting is not the bottlenecks, rather the action efficiency and the uh, accumulation of the knowledge over a very long context because you need millions, hundreds of thousands, if not millions of tokens to solve this. And keeping consistent knowledge of everything that has happened over such a context is a major engineering challenge at this moment.

Speaker E: What is harder um, in arcv3 compared to the other one is this interplay between exploration and solving the game. Because um, in arcv1 and arcv2 you would get all of the information in a static frame, um, as you are given the puzzle instead in Arcv3 you are given the game, but just from the first frame you cannot understand what needs to be done. So you need to start interacting with the game and through that interaction you gather information of what the game is about and you start understanding how to solve it. And at the same time you need to try to understand what the game is about and try to solve it at the same time. And this interplay is very hard to kind of um, explain to the agent how they should do it in an effective way such as general and generalized to across all games. So I think this is probably one of the key part that is hard about Arcv3.

Speaker A: What we want is abstraction based exploration.

Speaker E: Yes. And that's I think the very hard part. And I guess here is where the similarity with Arcv2 and 1 are, that you kind of get the right abstraction level at uh, which to do this exploration and kind of understand that oh, the thing moving around is probably an agent and that I can kind of control. And it's not just kind of patches of Pixel moving around. And if you're able then to get this right abstraction then it's usually kind of trivial to solve the game. But uh, yeah, making the agent acquire this right abstraction level, it's the main challenge.

Speaker A: How difficult is it to acquire the goal? Are you finding that these 27 billion parameter models can just reasonably infer what the goal is or is it much more complicated than that?

Speaker E: It some in the good runs when they get the goal right at the first um, at the first trial it's very easy. They kind of can. If they find the right hypothesis for the game then they would go on and solve a lot of levels. Um, but if they maybe try once, make it wrong and then have give another hypothesis um, and don't and it's not the right hypothesis again, then it's very hard to get them out of the loop. And we often find that they get stuck in very kind of uh, not intelligent goals that it's very clear they're not right. For instance, often the agents start thinking that reducing the energy bar to the minimum is the goal or that stepping 10 times in a region is the goal. Which for a human is kind of clear that it's not the actual goal. But it's very interesting that uh, they're not able to see that there's no way that's the actual goal.

Speaker A: Yes. And can we contrast rewards and goals?

Speaker C: We can use various different things. Like the most naive implementation is just use level transitions and then the ARC AGI score for that specific level transition. So if you did offers. Yeah. If you just worse took more actions than what a ah, human would have taken, your score would be lower but you use that per level. Um, and then you can add various other types of rewards to improve your objective.

Speaker A: And did you use reward shaping?

Speaker C: Yes. So for our um, RL training pipeline we do reward shaping. We have 25 games. We actually have uh, a lot more games that we've generated ourselves and we can for example train on them to make the agent better. For example exploration, finding goals, achieving those goals. Um, yeah, through end to end RL on just a bunch of games basically. And the thing is this is much more difficult to do than Ark 2 arc 2. This was the standard approach. You would pre train on a lot of uh, puzzles and then you would do test time training as well on the set of puzzles. But to do this in Arcgi 3 you need to train over uh, as Michal mentioned, like 100,000 uh, 200,000 uh, tokens, uh, which is extremely difficult to do so we try and improve uh, this by for example adding reward shaping, um, training on shorter sequences, trying to generalize to longer sequences. Um and I guess what makes this so interesting for two for labs as a problem is that this is exactly the kind of research that is currently set of the art agentic multi turn, um, long context reasoning. And uh, yeah it simplifies the problem. We don't have to worry about environments or other complexities like safety. We can just focus on the core problems and try and see if we can make progress there. And that's also perhaps uh, some area we could actually contribute some new research um, which might uh, I guess uh, be open source at the end of the competition or we'll see what we want to do with that research. Um, but yeah basically on your reward shaping question we add various rewards level rewards, archeologist scores, whether the code gets executed or not, um, the length of the uh, reasoning steps because if it reasons too long per step it might take too long to actually solve it and you only have a certain amount of tokens per game. So all of that we try and um, improve.

Speaker A: You know the weird side effect of Archv1 and 2 is it left me with the impression that intelligence is overrated. And what I meant by that was, you know, yes, we could solve problems, we could do hill climbing, you know, when we knew what the problem was. And maybe you would agree that in Ark AGI 3 when we know what the goal is it becomes a hill climbing problem. But it feels like the challenge is abstraction. And it's almost unfair that Sholley is doing this action efficiency thing. I mean in my opinion what he should do is um, yeah, you do it the dumb way the first time and then you compress that into reusable knowledge and it's in your library. And then the next problem that comes up, which should ostensibly be using what you just learned in the first game, then you become more efficient. So you kind of become more efficient over time because you're compressing knowledge. That kind of feels like the goal to me.

Speaker C: Yeah, so there's actually been discussion in the community about let's say for ArcadeJ4 having that as one of the goals. You play a game, you solve it in like a large compute or action budget and then you have to play it again and you have to do like a speed run through it to improve it. So I think that is important. But I also think action efficiency is a practical step to reduce just brute force solutions. It counteracts that you have to explore in a more Intelligent way and kind of makes sense in real world environments as well like computer use, uh, coding agents. You can't just explore every possibility, um, you have to do it in an intelligent way and improve uh, using some heuristic. And I think it makes sense uh, to not always have uh, just all the actions available to you. Uh, okay.

Speaker A: But even on RKG3 though, when, when you do have the goal and it's a hill climbing problem, I guess you still can't brute force it because you get dinged on efficiency. But in principle you could, if you had enough computation you could.

Speaker C: But even that they made so they updated the games to be more robust to that. So uh, like level one you could in principle brute force in a certain amount of actions. Level 2 the computational power you normally need is enormous. So you can't really with a reasonable compute budget even get past level two. And then you have level three, four depending on how long the game goes. So yeah, in principle you could brute force anything. Um, but practically with the compute budget we have, you won't be able to do that.

Speaker A: And could you just describe what is it about arcadegi3 that makes it almost impossible to brute force?

Speaker C: The main thing is uh, the action space. So we have eight main actions but there's a mouse clean action which has around 4,000 possible places you can click 64 by 64. And also the amount of actions you normally take is quite large. It might be anything from a few hundred to a thousand actions on all the games. So that branching factor is just enormous if you're not doing that in an intelligent way.

Speaker A: Is Anything in Arcade GI3 badly designed or gameable? Because um, didn't the preview have some. It was a bit brute forcible wasn't it? You were talking about that earlier.

Speaker C: Uh, yes, yes, exactly. So I guess that was the main point about the preview competition to wish to show whether it's something you could easily exploit. And indeed there was uh, like the Stochastic Goose algorithm and few other algorithms actually show. I think the first four places was basically brute force algorithms that just search over a large space of actions but do some basic form of filtering and you can get a very good score. Like I could solve two games, almost solve the third game, um, on the private set. Um, the game is also too easy so they upped the difficulty level. And for example one other thing was the timing bar which only changed when you actually executed a valid action. So you could easily learn what actions are valid or not, um, on the new set of games and RKGI setup. I don't know if any of you have noticed anything that is easily exploitable or badly designed. Um, yeah, I think it's implemented much better now and it's, that's why the scores haven't shot up initially. It's still, uh, at like 1%. So I think it's still a challenging setup.

Speaker A: I guess the million dollar question though is do you think it's possible in principle to do really well on RKGI3 and be no closer to AGI?

Speaker C: Uh, yes, I do think it is possible. Um, but I guess that's the point of the benchmark as well, is that it doesn't have to prove that once you solve ARC AGI, you actually have AGI. But I think as the art prize has mentioned over and over again. I see just disproves that something that can't solve this benchmark is not AGI. Um, so, yeah, I think it's definitely possible, but getting 100% on this benchmark would definitely require some major improvement if it was just like a base model or something. But we're nowhere close to getting 100%, which is the grand prize that is also made available at the end of this competition.

Speaker A: And do you think people will get near to 100% anytime in the near future?

Speaker C: So I guess that's a debate. Um, some of us believe it might be possible. I don't think we're going to get that this year. Uh, but, um, yeah, I don't know if there's any other opinions on whether we might.

Speaker B: So there's, there's, I mean that's the, the, you can split the problem at least into like, will someone reach it in the Gaggle competition where you're playing 110 games and nine hours on a single RTX 6000 versus is someone going to do it with a frontier model? 100% is essentially impossible because it means you cannot kind of mess up once. If you kind of have the wrong hypothesis on one level, that's it, you're done. So I'd more like say, but, uh, no single human is likely to score 100%. So at more like maybe 90%, you could kind of say this model is capable of doing this. I kind of feel that is actually possible even within the constraints of the Kaggle competition. I do think it is going to take more research and probably simply better base models than will be achieved by November when the competition ends.

Speaker A: Do you think there's some notion of, uh, almost Kolmogorov optimal score? You know, like, if you were the perfect intelligent agent, you would kind of converge on this score. Does that even make sense?

Speaker B: Probably yes. And I do think that score would probably be 100%. Like the game should actually kind of be designed at that score is 100% that you don't kind of. That you don't have to. You kind of that score, if it's not 100%, it essentially means there is guesswork involved.

Speaker A: Didn't you say before, though, you could only get 100% if you already know. So like, if you were an agent that genuinely needed to acquire knowledge to solve the task, so. So you had to do like the minimum number of steps to acquire the knowledge, do you think that would converge on a certain score?

Speaker B: Yeah. So that. That's kind of what I think would be like 90% or something.

Speaker E: Yeah.

Speaker B: Simply because it is not reasonable to like, we just saw the Atlas 20 game, there's two places you can move to. Yeah, you just can't guess that. Right. Every time. The thing is, you're being compared to humans who also had to do the same exploration and you get median score. So kind of wisdom. If you're averagely lucky, you're going to do it. But yeah, you have 110 games, you're going to get unlucky in one of them.

Speaker E: Yeah.

Speaker B: And not yet. I will mention, even with our simple with Gwen, we can already get sometimes 100% on certain games. So that it is, let's say, certainly doable to beat those human baselines.

Speaker A: What we were saying earlier is that one view is that intelligence is this quite crystallized process. There's a concept of iq, for example, and if you have a certain amount of iq, then it's predictable how well you can generalize how efficient you are. And I guess the data might just not back that up. The data might just say, actually it's kind of not random, but very specialized. And there's huge differences, individual differences in capability in machines and in humans. And would that make them kind of reassess their whole idea of what intelligence is?

Speaker C: On RKGI3 specifically, I think we have a clear benchmark which we know humans, which is general in some sub domain, which we can say whatever that domain is, can score that score and that is the medium score. So I think Even just in Arcgi 3, if we do bad with our, let's say the final submission score is like 5%. I don't think that would update their views because humans can achieve that.

Speaker A: Or maybe they would argue that if it did converge and become more Regular. So if there's a new class of algorithms that kind of consistently solve the problems in some predictable amount of time, maybe Chollais would think, oh, that's the algorithm of intelligence. It's not just guessing anymore or something.

Speaker D: I think the goalpost is going to move further. I think they will update their, uh, model. I think in some way they already have by changing the games because they fought. Some of the games that they made, uh, weren't brute forceable and they actually were. Um, and then there were some simple models and they also changed the metrics, updated it couple of times, both to be more fair and to represent better human performance. So certainly they have been updating their model a little bit already. Um, it's tough to say what's going to happen, but, um, I think the goalpost is moving all the time and if we create an agent that would be able to solve all of this, then we're going to still discover new areas where it's still missing and it is not targeted by the data set. Maybe it's rumors, but I think there's like other Arc AGI competitions coming up. They're building Arc AGI 4 or something for next year. I think so. Um, certainly they have more to add to the story than just arc 3.

Speaker A: Do you think that they won't release 4 until people are doing well on 3 or they'll just release it anyway?

Speaker B: If you look so far it does feel like they. I don't know if it's intentional or just the speed which we developed them, but it does seem pretty clear. Alas, I don't think they knew what Arc AGI 3 would look like when they built Arc AGI 1. They might have even thought, oh, it's going to be 10 years before anyone beats Arc AGI one? I think you could.

Speaker D: It was like five years in the end.

Speaker B: Yeah, sure. But the point is, at that point you think, okay, now they can do this. Does this really mean intelligence? Well, probably not, but I think also you need to see what happens arc agi3 and also the ways in which it gets solved to then also understand where the gaps still are. Because again, no one is claiming if a game can play. Uh, if you can do a 6004x64 game, you're intelligent in every sense. It is just one more step and one more challenge that they've beaten. I can imagine. Maybe, for example, Ark 4 has kind of games like you're playing Doom, for example, but not Doom, but of course a new version of Doom that is a whole you need to kind of work with a much more vaguer representation of the world and it would be a completely new challenge again. But I imagine, yeah, it's going to kind of taking a little time seeing what's happening here. I have also no idea what they're doing in the background right now.

Speaker A: Uh, yeah, I've no idea actually.

Speaker E: Yeah.

Speaker A: Ah, Sholay was quite forthcoming about Arcade GI3.

Speaker C: I'm wondering if it's not the idea you discussed because the community has discussed this a lot and I think there's been some back and forth about uh, having to play a game and then having to acquire the skills and the knowledge and then replaying it at let's say almost maximum efficiency, like speedrunning it. So that's one intuition that's been going around in the field, but I guess only Greg and the team can tell us next year.

Speaker A: The thing I really liked about ARC, uh, V1 and 2 was that, I mean first of all it was a grassroots community and loads of people without lots of funding could just focus on the kernel of intelligence. And now it's just ridiculously complicated and it's almost de. Intellectualized the field which means that, you know, loads of folks with resources are just throwing reinforcement, learning, learning algorithms at it and almost going back into kind of blindfold mode, you know, which is, I think it's a fair description of a lot of machine learning. You know, like we just have really big models and we just train it to do next token prediction and we see what happens and we try and scale up. The thing that was so magical about the community was there were just so many great ideas and like, you know, just really, really deep thinking.

Speaker E: It's not that simple. Like even if we have a lot of compute, um, it's still hard to make a harness that solves these games and it's still and only starting and training is also hard. It's not um, as simple as just throwing computer. Still many details that you have to get right. But for sure, as you said, I think it brought up a lot level needed to enter the competition. We're seeing that many people are stuck in the kind of template solution and there are as far as we are aware, at least in the competition, not many trying kind of LLM approaches because it's so computationally expensive. So it's definitely made it harder for um, kind of the average person to enter the competition. But I wouldn't say that it's just a matter of throwing a lot of compute at it and brute forcing.

Speaker A: Yeah, it's not like most other benchmarks.

Speaker E: Right.

Speaker A: There's no language, there's no instructions. Does that make it quite distinct as well?

Speaker E: I think that's the interesting part about arc, that it kind of tries to remove as much as possible the prior that you get from language, um, or from human knowledge and kind of strip them to the minimum and really only test for intelligence, I think. Yeah, that's a cool feature.

Speaker A: Language is the basis of how we do a lot of thinking. And what you folks have done is you've gone to language and then gone back again. So we've got this kind of loop. So uh, we go to language, we do some reasoning and then we might use that to do active fine tuning or like, you know, do reinforcement learning or whatever. And we've got this virtuous cycle. So he took the language away and we're kind of putting it back again.

Speaker C: Yeah, I think the reason we're putting it back is because we're specifically focusing on language models which has been extensively trained on language. I guess if there's other approaches like um, neural guided search, they might not use language at all. You don't need language for this. At least I have human textual language. But it's just, and in our case we're heavily leveraging, uh, reasoning models so it's just natural to bring them back into the domain they're experts in and what they're doing, uh, while we want to actually just harness, should shape that and encourage that uh, behavior. Uh, so yeah, I guess that's the main reason we're moving back.

Speaker D: I think it's also very difficult to like, I think the reason, and I've seen, I said it before already, is that the reason why language succeeds so well because there's like higher level priors that are encoded that are just so difficult to strip away from the data. May that be what? Uh, like the humans have created these games and they have an idea of what a game is and even though they tried to remove um, the biases a little bit, they're still there. You might look at these games and, and usually the backgrounds are some sort of like dark color or white rather than green. And this is like not necessarily true for all of them, but like more or less it holds. And you can see that this was created by humans and what their idea is of games. Um, and that is just why language holds. I think it's incredibly difficult for a human to create something that doesn't have this in it. It might be even impossible. Um, so that's maybe why Coming back to the language works so well.

Speaker C: Yeah, just quickly add on that. I've also seen exactly that. So one thing you could do, you could easily permute the game so you can permute the colors and also rotate the um, images. And when you do that the games become significantly harder. So if you just remove that priors, uh, which shouldn't actually be priors, you can actually see it becomes harder for humans to play. So yes, perhaps that tells something about the benchmark. There's some leakage of human priors into the game.

Speaker A: Harder for the algorithms as well.

Speaker C: Yes. So that's one thing we found of the harness. If you not so you can provide numbers as colors, it does worse than if you encode those numbers to specific colors that it knows like black for background, gray for immovable areas. So if you remove those priors it does perform worse, which is interesting.

Speaker A: That is fascinating on why I think

Speaker E: it's important to pass through language. Because initially we try to just train a uh, neural net with RL on a lot of games and kind of not use LLM altogether and try to get us prior in by just seeing a lot of games. And it's way, way easier to just um, use an LLM. Like you get away, like you get good performance much faster. Because um, the prior for language is so general. It can be applied uh, to so many different domains and getting the same prior by just training it would take like so many games instead, uh, by starting from language and then fine tuning to the specific games you can get there much faster.

Speaker A: You know it's quite a common uh, technique to transform something so that it falls into a representation that has more friction with our knowledge.

Speaker C: The encoding from just numbers to specific. Like you use chars, like let's say C for or like let's say use a B for blue and we can tell it like this is blue, this is this color. This automatically helps it a lot in language to reason about it because it knows that typically when humans play games, bright colors are objects you want to interact with or something's going to happen. More dull colors, background or walls or something like that. So that definitely helps uh, a lot

Speaker A: because that's actually really interesting because I suppose one strategy is you could have multiple representations or um, from a language point of view as well, you could do something really galaxy brain like turn it into a story where the colors become dogs or agents or people. Do you sort of mean like is there some representation that will just make uh, the language model come to life? And it's significantly More powerful.

Speaker B: As far as we've seen so far the models are actually pretty decent at perception because remember the game in the end is not a 64x64 grid. It has concepts, it has a player that's moving around, it has a goal, it has the maze and you're not going to play the game if you don't get those concepts. The thing is the LLM is pretty good at getting at those concepts. The problem is that it takes a while and we have very little time to play the games. So one aspect in the duck that we haven't discussed yet is for example that we give it different views on the world. So very much in Elski grid. But for example we also have a simply, it's very simple, it's just kind of uh, a uh, segmentation based on connected components. By just having that and giving it that list, it works well on some games, not well on others. But it kind of allows the model to short circuit a lot of this perception part. So in the perception it will kind of, yeah, look at Auski Gerrit and we'll zoom in on part of the board and make an Ausky Gerrit there. Oh that looks like this and do it on another one and spend a lot of tokens that way.

Speaker A: What about from a vision point of view as well? Do you think language is actually necessary for reasoning? Like do you think in principle we could train a vision model to go like directly to the output from the input?

Speaker B: I think it boils down to one of the big open questions in the field. Like is language critical to intelligence? I uh, will say, let's say at least when I'm playing the games myself and I think that goes for all of us. I'm using language like these games are set up that indeed with the object prior. Like uh, what a four year old could do it but I don't think a 4 year old would be beating most games. And I'm actually thinking through a strategy myself. I am thinking let's say out loud or say in my head but there's a narration, a narrative in language basically oh, we have to get the key but the key is blocks. How does that work and how you decouple all that? I'm not sure.

Speaker A: Uh, language is almost like an emergent phenomenon. So it's still almost determined by its physical instantiation and the world that we live in. But it's something which has evolved over a very very long time. So we could in principle train an end to end vision model and if it saw an insane amount of data, like almost all of the distribution. It could potentially acquire the same abstractions. But language is almost like a shortcut because you can bootstrap those abstractions.

Speaker B: Yeah, it's also tricky because it could, but it might end up kind of internally developing its own language because you need stuff like reasoning chains. I mean in our case it needs to write code which is a language itself.

Speaker C: That is a good point. So I did some initial RL experimentation and what I did uh, for the first experiment to show that we can generalize. So I created a ARC environment that's procedurally generated with some new objects and new objectives. And in fact you get the behavior it's just like a vision model and um, uh, like a Mamba backbone. And you can get that behavior that it learns to figure out the objects and place them in the right order. Um, just without using um, language, just using a latent uh, representation. Um, so I think it is possible in principle. But what that experiment showed is I required about 10,5000 different permutations just to solve that one setup on real ARC games. It's going to be more than that if you want to train from scratch. Um, so yeah, that's why the prior helps a lot. You need less environments to actually get it to perform well. You could just do it with a harness. And if you train the hope is that you need less environments to actually just improve on and generalize, which is the most important part.

Speaker A: Yeah, and it does lean on this more general point that we think, we think humans are so intelligent but we have the benefit of millions of years of evolution.

Speaker C: Right.

Speaker A: So we're kind of, we're using all of that trial and error implicitly. And so it's almost a bit unfair that we're expecting AI algorithms to be able to do all of that. Exactly how do you square the circle between. You know like you're releasing lots of pre built tools that are quite specialized and you also are bitter lesson pilled. Is that a contradiction? Like what does it mean to be bitter lesson pilled?

Speaker E: As a lab we subscribe to the bitter lesson. And um, I guess the idea behind, behind the bitter lesson is that um, often detailed uh and specific solution uh lose compared to just more data and more compute and kind of going large scale on a problem. I think looking at the past, even if the bitter lesson wins in the end, it's always useful to try first the more kind of detailed specific approach. For instance for um, Alexnet, uh, which was the paper um, that won Imagenet and some way Started the deep learning revolution. Um, I think a lot of the intuition that brought them to use convolution neural network, uh, for image recognition were in some way motivated to all of the uh, tricks that were done before to do image recognition. And so I think that even if it's true that now we're doing pure engineering on the hardness, we're kind of um, understanding the details of the problem that then will allow us to do a more end to end kind of solution, um, and allows us to do something that generalize hopefully more So I think this understanding of the details and specification is um, probably a necessary step.

Speaker B: Yeah. I want to add that I may be a little bit less bitter lesson built than most AI researchers also. Mainly maybe because I was doing it uh, maybe before the bitter lesson became the baseline. But I do think at least certainly where Arc AGI 3 is right now, I would kind of be willing to make a bet that the winning solution, be it also someone else's of this competition, will actually not be a bitter lesson solution. So if you look at ARC AGI2 now, it's been pretty much bitter lesson and it's being won by creating huge amounts of training data. I don't think there's a clear path just yet for this competition. And uh, what is needed now is really kind of really the yeah, basic design. I do think it will involve training, but probably not to the degree that all you need to do is have a good training data set and then go for it.

Speaker A: It's fair to say that Chollet does not want it to be bitter lesson pilled. Like every single design step has been away from the bitter lesson and he would, he would love it. In a world where it wasn't the case that, that it was bitter lessened,

Speaker B: I do say I might be willing to make that bet for this year if there's another iteration next year or another one the year after that. I could imagine that at some point it is bitter lessened, but I don't think the models are there to be able to do that this year already.

Speaker A: But yeah, it's so amazing that even in 2026 we have so much uncertainty about this. Um, we still don't know if beyond a reasonable doubt that it might not be bitter lesson pilled in two years time.

Speaker D: Important to bring things into distribution. And what Arcprice has done for Arc AGI 3 is that they specifically say that the training problems or the public problems are not representative of the testing problems. So that makes it even harder to create a distribution on which you could meaningfully train because you still have to extrapolate in some way. So maybe that's also why. And overfitting just doesn't help because you have to generalize to new problems. Um, so as you say, they are specifically targeting this not to be just like mass trainable, but maybe if we can um, extrapolate the distribution well and guess whatever, um, guess what's in the testing distribution, then this could be done. But this is just hypothesis.

Speaker B: I do think maybe in a sense it is going to be bitter lessened. But the classical bitter lessening is just you make a training set so big that everything is in there and then just bake everything into your weights and then you're there. Which is kind of what happened for ARC AGI 2. I think this one does need something like that, but it needs to have. And I think that is maybe much more what Chollet would also like to see what you need to get into the model is the basic thinking patterns. So it needs to know the general ways in which to approach a game. And how do you solve a game without having memorized every game in particular? And I think we can manage to do that. Yeah, we'll feel like we've really done something past just beat a bunch of games.

Speaker A: And how do you reconcile the fact that you're trying to build more powerful AI but you're worried m about it.

Speaker F: So yeah, reconciliation is I think uh, you cannot do AI safety research in a vacuum. You need to be clued in about the best techniques and being being able to kind of train good AI systems to be able to study them at all. The way I would say it is if you want to make bridges safe, it's going to help to know civil engineering and how to build bridges. Uh, and so I think similarly for AI, but that runs the risk into. Then you participate in the race and you make that um, kind of racing situation worse. Uh, because uh, this more, more like the progress about AI capabilities runs ahead of progress towards AI safety. So I want to be very careful with this. I think now I'm very happy with the team we currently have and the uh, kind of level of technical discussion and common interests and actually work being done. So that's all very positive and gone wrong. It just takes forever to get there. Somehow it takes longer and more expensive than what you think. Yeah, it will. And then I guess what's wrong is the broad AI progress is very fast and it feels um, a bit tough that. Can you ever catch up? I think the next year might not look that different than today, but I would say five, 10 years away might look completely different. So I think I'm a believer in the software only singularity. I think software advance will keep being made about kind of agent, uh, engineering, uh, stronger AI system in the world of atoms generally. Uh, but I think eventually we replicate this success with things like um, industrial robots like self driving cars. I think they at some point the movie Paddocks kind of falls and then we start seeing powerful AI capabilities everywhere.

Speaker B: Athletic Brewing Company crafts award winning non alcoholic beers for those who want to be part of every round.

Speaker C: With over 185 flavor awards, they're exceptional

Speaker E: NA beers that fit your lifestyle and any social occasion.

Speaker D: Summer's full of good times and Athletic fits right in.

Speaker B: Go to athleticbrewing.com to have brews delivered to your door or find them at a bar, restaurant or store near you. Near Beer Athletic Brewing Co. Fit for all times. The most memorable gifts aren't found. They're made. Zazzle is a custom marketplace where you pick any product, a mug, a card, a tote, a phone case and make it personal. A photo, a name, an inside joke. The kind of gift that actually fits the person. That's what 30 million customers have been coming back to Zazzle for over 20 years to find. Right now, save 25% on your first order@zazzle.com that's zazzle.com make it zamazing.