Anthropic Code Leak: A Rare Look Inside Frontier AI | EP.52

Welcome to Hidden Layers, where we explore the people and technology shaping artificial intelligence. I'm your host Ron Green. Here's today's question, what can we actually learn from the recent anthropic code leak? Anthropics is one of the most influential companies in the world. Its AI models are at the frontier performance and it's where it gets out of major influence on how people think about the future of AI systems. So when internal code leaks, it's more than just a security incident. It's a rare chance to eat a glimpse inside how a frontier AI company may be building and organizing its systems. In this episode, we're going to look closely at what leaked, what it reveals about anthropics and internal design decisions, and what broader lessons the rest of the industry can take from it. Let's get into it. All right guys, let's jump in. The leak, this is a pretty big deal. I was really surprised about it. I think this is proof that anthropic is moving really fast. As a side note, they went from I think their annual revenue run rate was projected to be about 9 billion in February and it had grown to 30 billion as we were recording this. I can imagine things are not over there. Let's talk about what was leaked first before we tease apart what we learned and what surprises we found. At a high level, the leak was about a half a million lines code. So this was not insignificant. It involved none of the actual model weights. So it's not like there was I think necessarily really deep core intellectual property that was leaked. But the harness, that's right. The harness around their agentic system and how it makes decisions and reaches out and does tool calling and things like that. So there's a lot to unpack here right off the gate. It's easy. You had an interesting tweet that you were going to share. Yeah, I was happy to see the leak. That allows people a view into one of the most advanced coding agents work. But I was worried after the leak, well, anybody get fired or is there an any setback on the iterative speed of the anthropic to release the clock code. But I want to share a tweet I really like. This is from Boris attorney who who who is the main author of clock code. And quote unquote, you know, it was human error. Our deploy process has a few manual steps and we didn't do one of the steps correctly. We have landed a few improvements and are digging to add in more sanity checks. And you know, he went out and say that, you know, like with any other incident, you know, the counter intuitive answer that they have is to solve the problem by finding ways to go faster, not, not slower. Rather than introducing more process, I really like that. Yeah. Instead of adding more process, they, you know, probably want to automate more exactly. Yeah. Take out the human steps. Yeah. So yeah, it's a, it's a lot of jam in there. And my biggest kind of impression, however, impression is that, wow, harness can be sophisticated. There's a lot of innovation to be had. I agree. Beyond the core model. You don't have to pre-train your model. Even have to fine tune your model if you don't want to harness. If you have a strong enough harness. Yeah. Yeah. I love that. I think we'll jump into that because one of the big takeaways I think is just that is how much is done outside the model itself. Yeah. Right? Okay. Michael, would you kick yourself? I think you were going to take a look at the memory system. And what do we learn? What surprises? Yeah. I mean, it's very sophisticated. And I think, you know, just to comment on the problem that's solving, obviously context is very expensive. The larger it gets is you have these like really long interchanges while you're generating code with the help of a coding agent. You need to figure out what's worth keeping around in terms of that context. Like tools you've used. Marked down files that tell you how to use X, Y and Z, whatever it ends up being. You have a very structured way of keeping the things that are important. And importantly, getting rid of the things that you don't want to keep around that you don't really care about. LS commands when there's a directory of like thousands of files. You don't need that in your memory. It's a waste. Right. And I thought that the way that they solve that is on the one hand, like almost seems overly simplistic. But on the other hand, it is really sophisticated. So they have a three tiered memory system. At the very top, there's stuff that's always loaded, no matter what. And that's things like your cloud that MD file, that in its entirety is always in context. And that's why it's such a special file in your repo. There's also a special memory.MD file. That is internal. So if you go in your dot cloud file in a project and you go down this nested directory, you'll find that memory dot MD file. And you can manual inspect it. That's an important piece. It's a it's just natural language. It's just a markdown file. And it's always going to truncate it at 200 lines. So it's extremely efficient. And that's the piece or at 200 lines or 25 kb's of context, whatever's first hit. So that's always there. And then there are things that are loaded on demand. Things that that memory file might index into like it might reference here's something important whenever you're making API calls to this very specific service. Oh, so it's all done. So the memory file can actually reference things externally that need to be pulled in as a part of the memory that are dynamic. Yeah, yeah, that's exactly right. And those also can be it mean cloud maintains this. It's inside that memory directory. But those are things that are loaded on demand dynamically. And then the third layer is this searchable kind of thing like full session transcripts every single part of your exchange. You know, it can search by keyword and pull things if it needs to their tools for that. But something that I thought was interesting is, you know, a couple of years ago, you'd imagine, oh, memory, let's just build a rag system. Let's make a vector database. But I think the important thing that they came to the realization of is, cloud needs to update and maintain this thing over time. Vector databases are almost more for like large tropes of information that you just need to accurately retrieve from. And it's highly interpretable as a human. Like you can go inside that directory, look at it and check it out. Even still, over time, I promise I'm almost done. This is great. Over time, there are issues that come up here. There's natural duplication of information, things like user preferences that come about during your exchanges, things like that, things like just unnecessary information that you don't need there. And I think one of the biggest feature leaks that came from this code base leak was that they had this concept of dreaming where in the evening, basically, you can have it run this async job where it'll go look through its memory. It'll do all that work of like deduplication, et cetera. And you almost have this like automatic, it's memory, but it's like, it's like a knowledge base that commemorates your entire exchange within the context of a project. And I think that's awesome. Yeah. And the analogy is draining is really strong because it's like you sleep, your brain is refining what you experience during the day, maybe saving core memories, you know, to sort of personify it a little bit. And it's amazing that they're doing a lot of that themselves. Another thing that I really loved about this was just how human readable it all is. Right. It's not some some really complex hierarchical approach. It's really just for the most part. Just mark down files. Okay, that's great. So I took a look at the skills part of the leak, which I'm really, really excited about. And so let me explain what a skill is really briefly in case somebody knows. So a skill is essentially like a procedural bundle where you can add new capabilities to the model. And what's really, really nice about this, just like the memory, it's simple text files, it's marked down, and you can augment the model and add new capabilities that it may have never seen during training. So this is like a really, really simple way to add new behavior to a model without having to train or fine tune it down the line. And it's it's arbitrarily extensible, right. So an example, an example of a skill would be something like you could literally create some examples of how to create presentations for your company and what what types of styles or colors or branding guidelines to use. Or let's say you had an internal code base and you had libraries that by it certainly could never have seen during training, where you can just document how this how these libraries and frameworks work and provide descriptions on when to use it and then cloud can reach out and and access those behaviors when it needs it. So it's a really it's it's almost it's almost unbelievably simple and powerful, which is I always think sort of the best intersection for technology. Okay, so what did we learn about skills? I was so excited about this. Well, my my first tech way was just kind of how little magic was involved. It works basically like you think. You describe a skill. There's there's metadata and it it's not like it has some really sophisticated classifier that it's using to organize this. It's just looking at your descriptions and based upon it's understanding it invokes those skills as necessary. So it's it's really really in practice as simple as you might have thought it could be, right, which which is just beautiful and it's like your memory stuff. It's just unbelievably simple. And the second is that the metadata is really central. So if you're going to build a skill, you got to make sure you describe that skill and when to use it and how to use it really, really clearly that if you get that right, then it will know when to reach out for that skill and it will know how to leverage it. And so just like with human documentation, it comes down to being really, really clear and concise and that's it. That's the bulk of what is going on, not a lot of magic. Yeah, I wonder whether that's in the front matter. Like when to use the skill was the trigger where I want to whether that's in the front matter of the mark down or is it in the men text off the mark down. But I think there is a so I built some some skills for like I'll give you an example. We have a I'm trying to build this knowledge base that's just stored in Google Drive that has some conventions that are important if you want to go efficiently search it for information. And I built a skill for it and it has that short description. And ironically, there's a skill builder skill in cloth so you can just have a conversation with it to build the skill. So whether it's front matter, I think it is just like a tag at the top of a single mark down file that you say, Hey, it's like a header or there's some way to does it convention to denote that this piece of text is the is the description of the skill. And for all skills at all times, I think all of those are in memory, those take up your context window. But to your point, dynamically, they'll decide when to load the entire skill in the context if it's relevant to that right in exchange. Right. Exactly. All right. Now ZZ you focused on compaction. And so this is really important because we don't have we don't have unbounded infinite ability to put in context with models. So could you maybe describe describe the problem and how they went about solving it? Yeah, I was super interested in compaction and how code does it because in my day to day work, I, you know, when I build agent, this is a pen point where, you know, some, and you know, whether it's document extraction or some other work where I need to gather a lot of data and and pass it to LAM to reason about it. But then the context is too big. You know, all these LAM has a limit of context, even the most powerful models has the, you know, like a 1 million context window. It's still, you know, small compared to all the context that you have. Right. So if, if we think about the kind of the core interface of like OpenAI API or Anthropic API, it's really messages in and then text out. So messages are like, you know, user message, tool call results or system message or the assistant message, right? And basically a list of messages, that is your input to the LAM call. And then you get back another text which is appended to your messages list. Right. Just grows and grows and grows as you get. Right. And this grows and grows and, you know, sooner or later, you know, it's pretty quickly that it runs out of the context window that your LAM can accept. So like, how do you actually compress it? Like, you know, not naively, like without looking at the cloud, source code, the leaked source code, you know, I would think, okay, just pass it to LAM, you know, to compress it. Yeah. And tell the LLM to compress it and give you back a summary. Right. And maybe in chunk. And, you know, for folks building like RAC systems, you know, there are ways to use vector DB and stuff to build it. So with that kind of questions going into the code base, I was actually, you know, expecting more sophisticated ways to do it. For example, building a vector DB, doing costs on similarity, you know, but that is not true. Their solution is actually, you know, much simpler and I think more, more elegant. It's like, okay, I think it's part science part art, where the compaction of the long list of messages into a shorter summary happens in kind of two major ways. One does not evolve calling NLM. So that's what they call micro compaction. So what that does is mainly, you know, all these toolcalls like read a file and pronounce the results or, you know, other toolcalls results, these are long. So the micro compaction does the deterministic logic to trim those, to snip those. And, you know, but still keep enough information. I don't know how they decide what to keep. But, you know, let's say I'm kind of curious. So it's deterministic. So is it deterministic just by length or, you know, so it's purely controlled. So yeah, I see this tension or, I mean, this major theme of these coding agents is, you know, the tension between the more dependable deterministic type script code or whatever code versus the semantic layer where the LM controls and makes decision. And how do you do the division of labor between these two? So micro compaction is totally deterministic. And it's like, you know, looking at the tool result, just strip out the results, but keep the request, but keep the toolcalls out. Oh, okay. Which is short and also informative of what has already been tried. Okay, that makes sense. Yeah, I think I'm oversimplifying. There are some more nuances in terms of what to keep and what to strip out. But, you know, right. So micro compaction makes, you know, that runs more frequently before each call to the LM, which I think makes sense because LM is expensive to cause. So that runs more frequently. And then the second type of compaction is full compaction, which is, you know, you still use deterministic logic to determine which of the old messages to compact and then add a boundary, add a compaction boundary. And after that boundary, all the messages are kept in full, but before that, and you need to calculate this very meticulously, like, you know, how many to compact, you don't want to compact too many messages, but you don't want to compact too few. But once you decide that you just passed that to the back of an API, the LM API, which returns a text to summarize what is talked about in that previous. And then that works like we would have thought right now. Yeah. And I didn't see any like knowledge graphs. I didn't see any vector DB. I don't know, you know, whether they have done extensive experiments and decided that, you know, those are not super helpful or is that going to be a future direction after a year or two, but I just found this interesting that their current solutions is fairly, like, you know, depending on simple mechanisms. Yes. Yeah. I mean, for me, the big takeaway, and I'd love to hear what you guys think, but for me, the big takeaway from this was, it's a fairly simple approach to these complex problems, right? Skills, as I talked about, that seems like something that is almost too good to be true, where you could just describe behavior. It's almost too simple to work, but the second I heard about skills, I instantly realized this was going to be a big deal because it's so simple and so elegant and so extensible, right? And that's what I think all of these have in common is they are, they're basically focusing on trying to solve these very complex problems in a as simple a way as possible, even if that means they're leaving some performance on the table, because that simplicity is really important at scale. And they can, you can always make things more complex. It's like, oh, the old saying like, no large complex system ever started that way. It always started as a small, simple system and then grew into that complexity, right? Yeah. Any other takeaways you guys want to share? You know, when I think about the experience of being a user of clock at like, whisper away all the the code leak everything and I think about it, it's the most seamless, fun, awesome experience as a developer. I'm a good fan of it. So clearly something under the hood that they're doing is special has some special sauce or it's really, really great. And I think if you do start looking under the hood, there's this, it's almost like they have this generic framework that's prescriptive about the kinds of abstractions you need for any sort of human in the loop, a genetic tool. So I think it's bigger than just coding agent harnesses. It's more like, this is the first concrete empirical pragmatic blueprint we have of how to build an agentech harness at scale. And I would imagine that, you know, obviously their workflow is going to get refined. They're going to change a lot over the years. But I think it's going to see a lot of really cool agentech workflows and tools that will come out and in the next year, I'm excited about it. Yeah. I completely agree. I think that agentech coding agents are the clearest proof that agentech AI is delivering real value now and is the future. Yeah. I'm really excited to learn from the cult harness to see how we can apply to help our clients building their own harness for Mexico, for document understanding, for retail. I believe those may need harness that's fun-tuned to each use case. Yeah. I totally agree. I'm sad for anthropic that the leak happened, but I think NetNet, this has been really beneficial for the world in a positive way. I think they'll be all right. Yeah. I'm sure it's been how many weeks, maybe a couple of weeks since the leak earlier. I bet the current version of the code is far beyond this leaked snapshot version already. Yeah. I think they're going to be okay.