Anthropic Accuses Alibaba of Distillation Attack on Claude

Speaker A: Today on the podcast, Meta is changing its mind on reassigning thousands of its employees to AI training. And we also have Anthropic, which is accusing Alibaba of a, uh, $28 million query distillation attack on Claude. Basically where you ask Claude, you know, 28 million questions, you get them back, you take the questions and answers, and it's called distillation, but you use them to train your model. General Intuition has just raised $320 million at a 2.3 billion billion valuation to train AI agents on gameplay. And Claude's paying customers are up 75% since January. This is pretty big because they're starting to eat severely into ChatGPT's lead. We also have Accenture, who is rationing AI tokens because their employees are burning them on basic tasks that are not super important. And Naveen Rao's unconventional AI claims a 1000x cut in inference power, which, again, I'm super excited anytime we can cut the water and the power, power used for AI, because that is basically how we make this scalable and we unlock a lot of really cool potential. If you've ever been sick of having to pay for multiple subscriptions to multiple AI models and you want to talk to all of the top AI models in one chat, this is something that always annoyed me when I was stuck with Just Claude or just ChatGPT or just Gemini. You know, I wish I could get the responses from others in there. Especially when you have something like Claude, which doesn't generate images, or you have something like Grok, which might not be as good at generating audio as 11 labs. I. I've built the AI box playground, my very own startup, where you can talk to all of the top 80 models in one place. So you can ask Grok that question that Gemini refuses to answer, and then you can get 11 labs to create an audio file for you. You can get ChatGPT to do deep research and Claude to help you with reasoning. You can do it all in the same chat. So if you want to go check it out, there's 80 plus AI models. There's image, audio, video, even music models that we have in there. Um, and you can talk with all of them in, in the same conversation. Start multiple conversations and you can compare the results side by side. Something I love is asking, you know, one model to generate an image and then asking ChatGPT to generate the image, and you can see the image results side by side, pick what model is better for which task you're working on. If you want to try it out. It's AI box, AI playground. It's only 8.99amonth. And I will leave a link in the description to go check it out. Okay, let's talk about what's going on with Meta. They have reversed the decision that they made last month to redeploy thousands of their engineers. And they were going to do this to a whole bunch of different AI training roles. They had this internal memo that leaked. It was seen by Business Insider, and it went very viral. But basically it said that they are going to, quote, defer to each individual's choice when they're talking about if they want, you know, people to go and work on the AI effort. So in the, you know, before, they were like, okay, look, we're. We're making a whole bunch of our employees are going to be forced to kind of go and work on this, um, AI training. They were reshuffling how they were organizing their whole company. And I think they got a lot of backlash from inside of the company. Apparently. Their cto, Andrew Bosworth, said in a, uh, recent interview, he said that it was, quote, probably one of the worst, uh, it's ever been. When he was talking about the morale over at Meta, just because I think they'd done a bunch of layoffs, they're making a bunch of shifts. Um, and so anyways, they had this big backlash. They had a whole bunch of people inside the company that were, like, signing up petitions against this big move. And so now they're going to let people decide if they want to work on the AI effort or not. My only thing that I will say here, because this was about 7,000 employees that were, um, kind of recruited in May that were going to be doing, like, data labeling and a bunch of other mandatory AI stuff. Um, I'm, you know, they. They have the option to do it or not, but I'm sure there's some percentage if they say no, they just will be let go or something. So I'm curious to see how much of impact this is. However, I think just giving people the choice is definitely something that will have a good, uh. Like, it's. It's good overall for the morale, but I don't know if it's like a really, like, oh, man, look, men is just letting them do whatever they want. So I think there's a little bit of that. Okay. Anthropic is accusing Alibaba of doing 28.8 million questions in a distillation attack on Claude between April and June. They said they had about 25,000 fraudulent accounts. This is the largest known distillation attack on the company to date. They said, and they kind of disclosed this in a letter to the Senate Banking Committee, um, that there is a major loophole. They said that foreign labs can bypass U.S. chip export controls by simply scraping model outputs to train their own AI. Okay, there is so much to unpack here. The first thing is they're like, they're bypassing US Chip export controls. Okay, forget US Chip export controls. What they're really saying here is anybody can do this, like Alibaba, whatever. Anyone and could go train a model, could go make that model open source. But like, what is Anthropic's m moat if Alibaba can go and run 28 million queries and distill their model? Um, forget about the US Chip export controls. Like, is Anthropic really saying that their model training is so fragile it could just be copied by doing that? You know, the billions and billions they spend could just be copied with 28 million queries. And if you think about it, you know, 20,000 fraudulent accounts. Let's say they're, they're paying $20 for each of these fraudulent accounts. Although likely they're, they're paying for the API, but let's just say it was the $20, you know, um, a month account. We're talking about, you know, $500,000. Like, really? So for $500,000, they could, they could pull something like this off. Now, I think it's probably more, but I mean, let's just, let's go 10x50 million dollars, right? Let's say they spent 50 million dollars in API tokens. Like anthropic is going to the government, it's going to the Senate Banking Committee. And it's like, oh, no, like these foreign actors are doing this. And I think even, uh, Elon Musk admitted in his recent hearing with OpenAI that Grok and Xai had done some distillation, like distillation. Model distillation on OpenAI. So everybody's basically doing this. And, uh, anyways, it's fascinating to me that they're so concerned about this because they're spending billions to train these models. And someone could come in and spend 50 million, let's say, do distillation and have a model almost as good. That's pretty crazy. Apparently, this particular attack was way bigger than Deep Seek, Moonshot, Moonshot and Minimax, who all did distillation attacks, allegedly to Alibaba, um, back in February. So this is kind of the Biggest one, it seems like all these Chinese companies are like, hey, well, forget the US Chip exports. We're just going to go distill Anthropic. The Trump administration separately ordered anthropic to suspend Fable 5 and Mythos 5 to all four nationals because of national security. But they're saying, like, look, um, you know, we, we had to suspend Fable 5, but like, let's say they didn't suspend it or even in the amount of time where it wasn't suspended. Uh, what if Alibaba went and did a distillation attack on Fable 5 and then Anthropic, you know, the Trump administration forces Anthropic to suspend it, but now, you know, Alibaba has it distilled and now they have that capability and we don't. All of this makes me quite bullish on open source models in the future because I, I mean, perhaps Anthropic's model's just gonna get, you know, exponentially better and better for all eternity, but assuming there's ever a plateau, it's gonna get distilled and put into an open source model and people, if they have the right hardware, won't have to pay. Anthropic. Although with the price of, you, uh, know, memory and everything going up like crazy, if you saw Apple had to boost the prices of basically every single Apple laptop by at least $100. And for the pros it was like $300 because parts are so much more expensive and there's such a shortage. So maybe even if we were all using open source models on our own devices, we're all gonna get hosed in the future anyways. Only time will tell. General Intuition has just raised $320 million at a $2.3 billion valuation to train agents on gameplay. So specifically video game footage. This is a company out of New York. They said on Thursday that Koshla Ventures led their round. They had General Catalyst, Jeff Bezos, Eric schmidt, a former F1 champion, Nico Rosenberg, and researchers at Google. DeepMind and MIT were all writing checks into this. I mean, if you got the researchers at DeepMind and MIT, you probably know you're onto something pretty solid. Um, this is a pretty big round. They actually did a round of funding back in October of last year for 134 million. So total they've raised $454 million. They're kind of a famous company because General Intuition fine tuned a robot using eight minutes of real world data. And then after pre training on gameplay from their company, Metal, there were, you know, the robot was actually able to walk and move. But they're using this data from like video games to help robots in the real world. And this is where they're kind of becoming um, you know, this is what they're getting famous for. The reason why this one in particular is exciting is because if this works then general intuition is going to be kind of the, the model layer that runs a whole bunch of robotics and simulation and gaming startups. So the same way that Anthropic and OpenAI and you know, all of these kind of general models are running, all the startups today that do software, this will be doing all of the robotics startups. So that's why it's exciting. Claude's paying customers are up 75% since January. This is a huge jump. This is according to a bunch of transaction data from 28 million U.S. consumers. And um, this is the first serious jump and I guess you could say competition to ChatGPT's subscription dominance because right now everyone knows, okay look, Claude's doing really well with um, with enterprise. There's a lot of enterprise players paying anthropic a lot of money. Um, and I think it, you know, slightly is beating out OpenAI there. But OpenAI always been the king when it comes to consumers, right? They're, they're the ones that they're, they have the cheapest model, they were came out first. They have kind of the most usage. But Claude is getting more and more popular with that demographic as well. If you go look at um, one of the indicators when people are talking about these metrics is there's I guess a bunch of like courses and so these are kind of self directed learning courses. And if you look at those, the Claude course demand is way higher than Chat GPT. It's actually three to one. So there's a huge shift. And I guess you could say, well maybe it's because it's newer and so many people already know how to use ChatGPT now they need you to use Claude. And to be fair, I think Claude has probably more to it for the average person. Right? We're all used to talking with ChatGPT. I'm not going to go take a course on that. Um, although maybe some people will. But when it comes to Claude and it's like, well there's Claude cowork and there's Claude code and there's like all these different ways you can integrate it into the, you know, building tools. There's a lot of courses that I think people would want to take and so I think this is probably why it's spiking so much. Apparently if you go to Data camp, they said that their users have searched for the keyword CLAUDE more than the keyword AI. And they said that the demand for Claude courses is up 18x in the last 30 days. So this is kind of a really big trend. And if we look at even the usage of um, you know, enterprise, we're seeing just a huge upswing. So ChatGPT is still the leader in kind of the absolute amount of paying customers that are, you know, using their, their model. But there is some really big growth coming out of Claude. Accenture is now rationing employees AI token access because apparently a whole bunch of their staff burned through all of their budgets converting PDFs to slides and running a lot of other really low value tasks through these super expensive, um, models. Now it's interesting and this is something I talk about a lot on my show. Just because you can do something with AI doesn't mean you should do something with AI. There's a lot of tasks where we just have very basic software that we've had for a long time and it, you know, gets the job done. You do not need to use AI to do it. This is one of the cases, right? Converting a PDF to slides. There's probably a lot of other ways to do this that are using the most expensive frontier models. And Accenture, uh, learned this the hard way. This is something that is, I guess kind of reversing a month's old mandate where they were, um, threatening people and saying, hey, look, you can't get a promotion if you don't adopt AI. And now everyone uses, uses AI for everything. And now all of a sudden they're like, oh shoot, like this is actually pretty expensive. There's uh, not a lot of ROI on some of these tasks that people are doing. So they're, they're kind of walking it back and changing their mind there. Justice Quack, who is the agentic AI strategy lead, told executives that unpredictable month to month AI spend makes it nearly impossible for finance teams to model costs, triggering cfo, COO and CIO level scrutiny. Earlier the earlier this year, Accenture built internal leaderboards where they're basically ranking their employees on AI usage to drive adoption. Uh, this obviously looks super counterproductive today. Uh, just saying whoever burns the most tokens on, you know, and maybe that could be on really low value work, uh, shouldn't be the people that are being, I guess, compensated the most or promoted the most. So right now they're now switching gears, they're going to start rationing and this is also coinciding with a multi day AI stock cycle sell off which is hitting a lot of memory chip makers. I think this shows that a lot of enterprises are unable to justify some of their inference hardware spending. A story I'm super excited about is that Naveen Rao, so he is the ex AI chief at Databricks, he left and he launched his own company which is called Unconventional AI and he said that an oscillator based chip architecture can cut inference power by up to 1000x versus a uh, GPU. So he has this new technology and he believes it is 1000 times one more product or more, you know, efficient, power efficient than a gpu. They just released their first UNO which is a working image model. It's basically matching Stable Diffusion's quality. This could be built at the perfect time when we're basically facing a massive crunch of not just energy but just, you know, having enough compute power with all of these hyperscalers. Sorry if this sounds a little bit technical, but oscillator computing uses coupled physical oscillators to settle into low energy states. And they're doing this instead of what we're typically used to with GPUs where they're shuffling bits through logic gates billions of times a second. So we're completely changing the way that we're doing the architecture here. And I think we have a real need because hyperscalers have this huge power wall, right? All of the big frontier labs opening I anthropic Google, they're all signing nuclear PPAs for gigawatts of generation and all of these, they're trying to get the energy generation from all of these new energy facilities but it takes so much time to build not just the data centers but also a lot of the energy generation. Right? The nuclear deals that Microsoft is signing, all of that isn't going to arrive till the late, you know, the late 2020. And that makes you know, basically any real efficiency gains that we can make today huge because that ties us over until a lot of these new power plants come online. A lot of the new data centers come online. We have a lot of, you know, uh, infrastructure that's going to get built out and we like, we're going to run out very quickly. I think everyone's realizing that. And so when we have these um, efficiency gains, it means a lot for the industry. So very exciting. Guys, thank you so much for tuning into the podcast and make sure to leave a rating and review so if you're on Apple dropping some stars on Spotify, it really helps us show out. It basically boosts it up in the algorithm. It helps more people find it. Um, and it helps me out a ton. Also, if you have topics that you really like that we cover or you want to hear more about something, uh, make sure to leave it in a review. I read all of the reviews. I read all the comments, um, and I really appreciate them and I help, I try to let those help guide the show to improve the quality on it. So if there's anything else you guys want, let me know in the comments and I will catch you guys all in the next episode. Make sure to go check out AI box AI as always, if you want to try 80 different AI models in one place for 8.99amonth. All right, catch you on the next episode.