The End of All You Can Eat AI

Speaker A: Foreign. Welcome to Raw Data with Rob Collie. Real talk about AI and data for business impact. And now CEO and founder of P3 Adaptive, your host, Rob Collie.

Speaker B: Well, uh-huh. Hello there, Justin.

Speaker A: Hello, Rob.

Speaker B: Recording this on a Monday. We're, uh, getting kind of ahead of things again. I like that.

Speaker A: We probably have to go back through the metadata of all the recordings to figure out, because most of our recordings for like, one big era were on Thursdays. And then recently we've been trying Fridays with some measured success due to personal schedules. Yeah, so now we're on a Monday.

Speaker B: Summer schedules have been difficult, mutually difficult. Hard to find us both at the same time. So here we are on a Monday. Not that the listeners really care. They're getting it on a Tuesday. Regardless, Jocelyn and I are going to go on our first actual couples vacation in, I don't know, 10 plus years.

Speaker A: That's awesome.

Speaker B: We're recording this so that we don't have an outage while we're gone. Perfect. And I really appreciate you making the time to join me on the off schedule Monday. I mentioned backstage that I'd really like to talk about this whole fable, the saga, the saga that is Claude Fable. And of course, the most interesting thing, the most headline worthy thing is the fact that it's been banned in a way that anthropic can't give it to everybody. The politics of all of this are the things that I try to stay out of. I mean, I wonder how much this is anthropic. Kind of like suffering their own marketing, right? Like, the hype around Mythos is so dangerous. It's almost like the Michael Jordan, the Air Jordans back in the 80s, banned in the NBA. So dangerous we can't even let it out. And then people are like, well, we'll just take that seriously. Even though you've trimmed out all of the places where it's supposedly dangerous. To me, the more interesting angle, the one that actually, like, kind of sent a shiver down my spine when they rolled it out for the brief, you know, five minutes it was available, saying, hey, through June 22nd, this will be covered by your subscription. But after June 22nd, any use of Fable, the Fable model will not be covered by your subscription and it will be charged on a per token basis. One of the things we've been talking about here for a while is how these $200 a month subscriptions are just like the greatest deal anyone's ever gotten. Even though they seem expensive to people who haven't really seen their promise yet, it's so funny. Like I've been like, oh, they've got a really hard problem. How are they going to transition all of these people like us off of these subscription models to usage based models. The whole time they were sitting on just an absolute ace card, which is we'll just declare the newest version of our things to be not covered by subscription. I saw like this like sort of like checkmatey type picture, like oh no, the free lunch is ending.

Speaker A: I remember when it came out, I was in the middle of a workshop when it dropped. And uh, like now everybody's got the new model and I remember saying to myself and to be in the room, I don't think I'm gonna try it because I'll get addicted to it. It'll only be here for 12 days. Of course I did try it. But interesting things on the whole subsidization of the costs. Right. So I mean I've done this for myself because you can actually see your stats of input output tokens from what you're doing in Claude. If I was just on the meter, I would be spending thousands of dollars a month and I'm spending $200 a month. And then you look at their other models for like teams and enterprise plans. They're all on the meter. Like if your company's on an enterprise plan, you're paying the API meter.

Speaker B: Yeah, yeah. And they didn't do that to the teams crowd yet.

Speaker A: Right. The whole thing is interesting because where this idea that we could stop all AI progress and the tools we have right now would be amazing and we'd spend the next however many years figuring out how to change our businesses and the way we do things. Well, that might really happen for some percentage of us where we stay at Opus 4.8 or GPT, 5.5, whatever you tend to use the most. And these better models, either they're more expensive to use or they're sort of out of our price points and budgets. And so we sort of just stay where we are and other players can kind of go, I'm just curious kind of what you've thought about that dynamic

Speaker B: because I'm like, ooh, yeah, I'm the same way. Right. So there's this other thing about model specialization. So when we fire up even opus, I mean that thing is loaded with everything. I mean it knows the history of the Roman Empire. I was on a podcast the other day that I don't think has been released yet where I was saying, without even searching the web, OPUS was telling me about the geologic Makeup of the ground underneath my house. Cause I was calculating the speed of sound in the ground.

Speaker A: Oh, from the pilings.

Speaker B: The pilings are back. And I was 100% calculating the distance to the pile driver based on the lag between the two. And I'm like, but hey, sound travels differently. Different. Different kinds of land. You know, we really need to get precise anyway when we're asking an LLM to help us with a slide deck or to make a judgment call on X or Y. So much of it of that LLM is just total overkill. The subscription based pricing model has kind of made this a moot point. Like, we don't need to be like, judicious. These models that have been built that are just about writing code. If it's just about writing code and understanding the interaction with the human being, that seems like a much narrower LLM than one that knows all these other things. The thing I say in my book, these things have a PhD in everything.

Speaker A: Permanent memory of everything.

Speaker B: And the other thing is that this LLM technology is famously not proprietary. Anthropic's models are proprietary. OpenAI's models are proprietary. But this is a genie that can't be put back in the bottle. I mean, subject of active academic research. Jamie, who you know is an advisor here at the company, has been laughing as to whether or not AI uh needs to become a lot more expensive or a lot cheaper. Where it is right now is not where it's going to be. It will force us, I think, incentivize us to be a lot more judicious about which club we're pulling out of the bag at what times. Like, whereas right now, why wouldn't you choose the biggest, baddest. I had the same reaction as you. Like, oh, no. The release of Fable immediately kind of made Opus 4.8 seem not great to me. There's this sort of psychological effect, but it's the same thing that I was loving. And I haven't really gotten all that scientific about how much better Opus is for me than Sonnet. You mentioned on a previous podcast, Opus 4.7 was a little bit wacky. I regret it afterwards, not asking you about that. But, like, in what ways did you perceive Opus 4.7 to be wacky? Because I didn't notice.

Speaker A: Yeah, Opus 4.7 was wacky. I think all new models exhibit this. They tend to have a slightly different personality. They talk a little different, they phrase their responses a little differently. You notice it, I think mostly when you use a model to do something on a Very frequent basis. So the best example for me is I've had a skill in claw that I've been using for, gosh, probably over a year now called daily planning. This is how I go about staying organized of what I want to get done. And I was, like, noticeably different the way it would talk to me. And this has happened at every model jump. And usually it's not that big of a deal, but sometimes the model will interpret my instructions slightly differently, and so it'll, like, miss a key detail. But the things I noticed with 4.7 the most is it actually started hallucinating a lot more. I remember thinking, I actually kind of thought this problem was solved. Remember, like, early chatgpt, like, holy cow, this stuff hallucinates all the time. Lots of like, hey, you said you did this. Did you actually do it? Oh, no, I didn't. I'm sorry. That's on me. So there was just. Seemed to be some jankiness in the harness. And I do most of this in Claude code that I found to be a bit annoying. The other one I joke quite a bit about is it would always ask me if I was ready to be done for the day in some shape or form.

Speaker B: Interesting.

Speaker A: And I finally was like, why are you. I had to, like, put it in my system Claude md. It's like, don't ask me if I'm ready to stop or move. Just like, that's not helpful.

Speaker B: I'll decide.

Speaker A: Yeah, well, it's like the personal interaction equivalent of, like, you. And I say, hey, Rob, you want to get together for breakfast? Oh, sure. We'll meet here. And we sit down, you order your coffee, and I'm like, it was nice seeing you. Do you want to leave? And we're like, what? We didn't eat? What are we doing?

Speaker B: Yeah, let's be done.

Speaker A: I'm enjoying Opus 4.8 myself. I did get a chance to do some pretty cool stuff with Fable 5. That made me go, okay. I'm not sure I could have done that with Opus 4.8.

Speaker B: All right, give me an example.

Speaker A: The main example is I ran. It was a fairly ambiguous task. I had IT run a comprehensive audit of our website's code base, which is something I've been meaning to get around to. And I was really impressed with the breadth of what it came back with and the correctness of it. This is something I had tried to do with Opus, I think probably 4.6. And most of it, uh, what it came back with, in my opinion, was sort of red herring type stuff. Not all that important. You're just, you're chasing things that don't exist. This came back with a pretty comprehensive and stuff. We actually went and implemented set of recommendations for accessibility, bunch of recommendations around SEO. It found some um, semantic errors in like our legal disclosures. Like for example we're registered as an llc, but it's like, hey, you're referring to yourself as an incorporation here. There's maybe 20 key findings and maybe one of them was bad. Okay, this is a new level of smart for that much context, right? Going into one thing and to fan this all out. And then the other thing I did with it is the Pathfinder product that I've mentioned to you that we're building. We had this big milestone feature set that we want to get through and I used Fable with the goal feature in CLAUDE code, you essentially set a success criteria, a test. CLAUDE could run itself and just say go until you resolve this. So you could say like, oh, work on this page until the longest content, uh, load time is X or whatever it is. And so I just use it to say, hey, here's this plan we have on this set of issues. Work on everything until either A, you finish it successfully do the criteria or B, you hit a roadblock and you leave a comment to what it is and I'll come back to it. And that was the first time I uh, stepped away for that much work in one go to have pretty minor catches and corrections on my end. And that was something I had not been comfortable doing with Opus. Like we're recording today, like I'm in the middle of working on some of this stuff right now. Like with Opus, I'm comfortable letting it go on some things, but I'm really meticulous on uh, some of the stuff that I know is important. I'm reading the diffs because I know how like terminology issues affect what ends up in the user facing versions of things. So okay, I've seen these couple interesting things with Fable, but I'm sitting here going, is that even something that's going to be in my toolbox in a practical way relative to how it's going to be available? Assuming it's available again, right the way it's going to be priced. And I don't think you said this in our text exchange. This isn't unique to Anthropic. I think the other frontier shops, they will have their next class of models might not fall under your subscription once

Speaker B: one of your competitors has done it. Unless you're going to be that company that's like going to bet on, uh, we're going to continue to capture market share. Like, I could see OpenAI saying, oh, oh, I see. Either way, this is good for OpenAI. They can choose to follow Anthropics lead, make more money.

Speaker A: Right?

Speaker B: Or they can choose to temporarily fake like they're taking the high ground and say, oh, we think that's dirty pool. We think that's bait and switch. When they're really saying, we think bait and switch comes a little bit later. So for now, we'll welcome all of those anthropic refugees who don't want to pay more than $200 a month for a Fable equivalent. I mean, I could see that be a strategic move as well. I mean, these people have been lighting money on fire with absolute, like, you know, just seemingly disregard for profitability. So, like, what's another few months? There's no way that this isn't the end game. I mean, this is where all the big shops are going to land on exactly this. It's just a question of when.

Speaker A: I do love the irony of the story. Dario saying, uh, oh, mythos, very dangerous, must protect the world. Also saying very prominently in one of his essays, we should consider slowing down. And then so, you know, everyone can read into the political situation between Anthropic and the. The government however you want to. And it's just. So how much of Is it political? Some. Right. The percentage you m. Form your own opinion. But okay, so if these models are so capable finding all these exploits and things like Mozilla Firefox that have been around for ages, been in the product for ages, Anthropic can't work on Fable. That's the other thing. So they just got Karpathy from OpenAI recently. He is not a United States citizen. He can't work on Fable. So, like their lead model, researcher guy, or, uh, members of their team, they can't work on it. So, like, they can't work on it, we can't use it at the same time, nobody else is slowing down. It'll be interesting to see how this all shakes out.

Speaker B: One of the marketing points is that, oh, sure, Fable is more expensive, but if it requires less time, you know, and less iteration to succeed, won't it be cheaper than Opus? The examples that you've been giving, you know, a couple of things that you've tried with fabled in the 15 minutes that it was available to you seem to at least partly support that hypothesis that maybe if you were paying on a per token basis for Opus And a per token basis for Fable. Maybe Fable even at twice the cost. Right. Maybe Fable would still be potentially cheaper because you'd spend less time and you'd chew less tokens overall, double the per token cost versus covered by your subscription. That is a big, big, big difference.

Speaker A: Well, it challenges the way we've explained thinking about model choice in the past, which is if you're on a subscription, like use the biggest, baddest, best available one you've got. We would say to people who are in companies that have enterprise plans, just like, yeah, you just use the crap out of that thing until someone comes and tells you to stop and then you figure it out. But now, so assume Fable 5 or something like that comes back. Now we do need to consider a decision framework of when do I jump over there? Because I think you're right. Like maybe not even only the time, but the fact that I could do something that I didn't feel comfortable doing saves me a lot of time, gives me a better quality of output. In what situations do I want to go do that today? I wouldn't use it as my daily driver because I have Opus 4.8 and a million token context window high thinking level. So maybe Fable 5, I think the chart was like at its medium thinking level was still better or as good and less expensive than Opus at high. I've already done the math. If I was paying on the meter for Opus, it was many thousands of dollars, not 200.

Speaker B: At what point do they pull that rug? Do they just say, hey, like Opus is going to be covered by a subscription forever? We've seen like with enterprise, they've just said there is no subscription. It's purely metered. It's interesting that they haven't done that to the team's plans.

Speaker A: Yeah, it's.

Speaker B: Yet.

Speaker A: Yet. Well, I think your comment about the gamesmanship between the labs about. Is OpenAI going to do something similar? Are they going to wait? Both of these shops are trying to, uh, go public this fall. So how they want to play that whole marketing PR game? Yeah, I don't know if everything went to the meter tomorrow across the board, that's a calculus that they're probably making. Is that net good for our bottom line and our top line and for the rest of us, like, okay, are we in the party anymore? Yeah.

Speaker B: How many thousands of dollars a month would you be spending right now?

Speaker A: Oh, probably six to seven.

Speaker B: Six to seven.

Speaker A: But we're small, right? We're just like a two. Like we're a small company. I would imagine for P3. Like, I could see so times that number by 50, uh, if not more. Right. At what point did business models really get forced to catch up? Put on our leader hats and say, okay, tomorrow, to use these tools now costs X. How do I recover that? In a way, you know, I think maybe that's where Dario is thinking about, well, how do we slow this down to make it all work? I think it's going to be a buckle up and get after it. Try and get good at this stuff as fast as you can. And I wish I wouldn't have tried Fable just yet, if I'm being honest.

Speaker B: We'll see.

Speaker A: And they said in there too, like, oh, maybe at some point it'll be available in the subscriptions. I don't know what your opinion is on how much of this is cost of compute versus scarcity of compute. They're similar, Right. But different problems.

Speaker B: Yeah, I'd love to see what they're like for their metered products. Right. Like what their actual margin is.

Speaker A: Can you imagine, like, uh, this is one of these guys having to go on like on a shark tank pitch, like, explain your profits while we buy it for $0.02 and we sell it for $1,000. Yeah.

Speaker B: And how much of it is CapEx versus OpEx? Right. Like the acquisition of the GPUs, the acquisition of the data centers to get an MRI done. Right. Is famously like, if you want to pay cash. It's famously like a $700 test in the United States. How much electricity is consumed by the mri? Not that much. I mean, even though it makes a hell of a racket, there's clearly a lot of voltage going on. Magnets, metal can fly across the room, so there's a lot of power, but it's still not $700 of power you're paying for the machine. The amortized cost of the machine over time. So, I mean, the margin on a single MRI in terms of just pure OPEX is close to $700. There's the technicians as well, of course. So on net. Right. Is it a bad thing if AI becomes more expensive? Right now, the threat of it relative to human jobs and stuff is somewhat elevated by. By the fact that it's apparently free at the moment. $200 a month is it rounds to free for what it's doing for you.

Speaker A: Most people's cable plans are more than that.

Speaker B: God, like YouTube TV can run that money. M. If we start paying sort of like the real price, but then at the same time, like the open weight models and the Companies start charging as much as they want to, it's going to incentivize us to evaluate less expensive options and it's also going to incentivize the creation of less expensive options. The market dynamics are going to change quite a bit.

Speaker A: When you look down your feed or whatever you subscribe to in terms of sources of news on this topic, there's a lot of competing narratives. AI is going to replace knowledge worker jobs. And then you see another study that shows like people are spending so much on tokens that it was actually more economical just to have someone continue doing a job. You're right about the incentives. I mean if we talked about the new Surface product the last time you and I recorded the local Devs AI product, let's say Anthropic came out tomorrow and OpenAI quickly followed suit. All of our models are now pay as you go. And I did the calculus of like what I want to be able to build and what our business needs to run. I'm like, okay, this is going to cost us a lot of money. I'm just going to go buy a box that's capable of running the best open weight option and I'm going to go that route. I would probably think really hard about that now. If I was in a different state as an entrepreneur, I might have more levers, right? Like okay, how do I change our pricing or our cost to serve? But like for me personally I would really need to think about that and I don't think that's unique for me at all.

Speaker B: I do think this also kind of uniquely precious in the development space. Yes, you're not running the meter to six to seven grand a month for your own personal usage on pre built agents that are doing specific tasks for you. Like you're developing software, you're developing websites, you're developing new systems and that's where a lot of intensive token usage happens. I suspect that for deployed real world AI like the cost to serve that Agentix systems that are customized and built for specific purposes, the price tag of those still might remain quite reasonable.

Speaker A: I think you're right about that because we've built a lot of that into our product and our cost to serve on those experiences is really, really low.

Speaker B: And you're already paying meter because that's

Speaker A: API, because you're trying to figure out, okay, how would you price this? And you're like, okay, like when it's contained in a uh, product, the expensive part is building things for sure.

Speaker B: Plenty of food for thought for the future.

Speaker A: Yeah, we'll see what happens and what other models do or don't get banned. And I don't like how in Claude code I'm actually looking at it right now because I just wanted to check something while we're talking. It still has my little Fable 5 currently unavailable.

Speaker B: I was going to ask. I'd forgotten about this. Like the real tragedy of all of this is the UX clutter that we're all experiencing right now. In some cases I can make it go away, in other cases I can't. You know it eats a couple of lines of readable text.

Speaker A: It does.

Speaker B: Uh, I want a credit back on my monitor. Like I need an extra.

Speaker A: You know there was a short lived bug that I don't know how many users has affected where the context window display that shows you how much you've views and how much you have left for Opus was only showing 250,000 tokens. It was early in the morning so I wasn't fully awake yet. I'm like rage searching on Reddit to find like what happened is. No, they roll back the context window too. Oh, uh, no, just a bug. Everything's fine.

Speaker B: Just a good old fashioned software bug. All right, well thank you so much. Have a good one, bud.