When Your AI Budget Hits Your Salary

Elijah Szasz: there are signs that the lifestyle subsidy is going to be ⁓ nearing an end. And I ⁓ what you about this. Have started to see this? Does this subsidy end with a price hike, a quality cut, with ads inside the chat?

This could go a dozen different ways, but I feel like... with history being an indicator of what might happen in the future with AI, that it feels almost unavoidable that something is going to change in this regard. how are you doing? Kevin Williams: Oh, I'm doing well.

You know, since, you know, 5.30, living the vibe coding dream, solving all of my problems. Everything is going swimmingly. Nothing is broken at all, I promise.

I mean, I struggle with this partially because I don't think that a lot of the more pedestrian applications of like chief of staff stuff, moving data around, grabbing data from other places. I don't think that has to be necessarily priced to the moon as the models get more efficient. It does not require the latest and greatest model to do these things. What it does require is a large amount of context.

Elijah Szasz: Well, very apropos as far as some topics I want to cover. I'm exhausted. I'm going to blame AI for how exhausted I am. yeah, broken things, vibe coding, all that stuff.

I want to kick this off with this term that's been flying around the interwebs, and it's called token maxing. Kevin Williams: and therefore a large amount of inference, which is what's driving most of the data center development out there. ⁓ there definitely will be costs, but I find it a little bit hard to believe that it's going to get like astronomically priced for the way that most people are deploying these tools. It get a lot more expensive, probably will get a lot more expensive ⁓ to very rapid development.

Elijah Szasz: Maxing in general is kind of an in thing to be doing right now. You can be looks maxing, can be gym maxing, you can be maxing all sorts of stuff. But token maxing, basically just shoveling tokens into the token furnace and watching them burn is happening for a few different reasons. ⁓ I you're pretty familiar with this as well, Kevin.

⁓ Kevin Williams: Again, sort of a weeks ago saying, you hey, I have an app idea, scope it, here it is, here's the scope, in your chair and it's done. Like you should be paying a significant tax on that. But I don't think the market has figured it out yet. And I also do believe that there are enough substitutes out there in terms of open source models that are reasonably good, dare I say Chinese models that are pretty scary.

⁓ yeah, yeah, yeah, of course. It's, it's, it's, are leaderboards in places like Meta of who can burn the most tokens, which seems superficially to be a really bad and expensive idea. But they, ⁓ those organizations at least are declaring that it's worth the dollars as far as experimentation and ego building and skill building, et cetera. Elijah Szasz: Yup.

Yeah, I mean, there's a bunch of stuff I want to talk to you about and it's all loosely threaded together. this going to kind of be the entry point. So ⁓ few different things people have said or a few different stats out there. There's a Stockholm engineer to the New York Times that he spends more on Claude than he earns in salary, like ⁓ employer actually pays him.

⁓ Kevin Williams: ⁓ you know, not suggesting that people dive into deep seek, like without knowing what they're doing, but there are a lot of substitutes out there and outside of the actual process cost, which is real and the energy cost, which is real, but theoretically we'll get more commoditized and more efficient over time for a comparable level of output of what we're seeing right now. I don't think you're going to get away with it as far as really jacking up prices and Elijah Szasz: know, that's interesting.

Just all these things, you wonder if these are telltale signs of a future to come where this is just going to be the new normal. Uber engineers reportedly burned through Uber's entire 2020 SXAI budget early in year, ⁓ like way Q1 was over, right? ⁓ Kevin Williams: Totally could be wrong. I truly don't know.

We don't really know. Elijah Szasz: The answer is we don't really know. And it's like, so little subtle things are happening out there. Salesforce just renamed the unit from tokens to agentic work units.

Right. And so is that a genuine value shift or is that just pricing camouflage? Right. And you're hearing a lot of things about, know, like Sam Altman saying AI is going to be like electricity.

This is, this is a utility. Right. And again, that's a conversation about pricing and how it gets delivered. Kevin Williams: So we need to retrench just a little bit here.

So most of our listeners are familiar with having a subscription. You might be using Claude, the $25 plan doesn't get you very far. So you're paying a hundred dollars for max or even $200 for max, or you're paying open AI, one of these subscriptions and a set number of tokens fits underneath that. Yes!

subscription and you will still burn through them. You've got to watch your usage a little bit based on what you're doing. So Claude design will burn through your tokens in a giant hurry. Whereas you can get by with like ⁓ Claude's sonnet for a very long time.

we're talking about here is the API based tokens as far as making calls to the models themselves ⁓ as doing more advanced development. Elijah Szasz: based on that pricing, Kevin Williams: So I mean, you actually read our notes. I'm so pleased. have a, you didn't.

Well, okay, this is what I wanted to talk about. I wanted to talk about Benioff ⁓ out. So ⁓ Benioff of Salesforce, there's been a lot of pivoting going on and we've been talking about these connectors and we've been talking about, ⁓ did I phrase it? The end of fiddliness that just loathe going into like a CRM because really at the end of the day, if I just treat this.

Elijah Szasz: I didn't, but coincidence go ahead. Okay. Kevin Williams: So smaller organizations tend to sort of keep their budgets under the $200 a month per head ⁓ limit because they can accomplish generally what they need. But if you're doing really advanced agentic based coding projects, it's really easy to stay like throw a cannon of tokens at these things.

And then you're being charged on a per million basis based on how smart of a model you're using. So if you're using Opus 4.7, Elijah Szasz: 40 clicks to find the one little thing that you want. Kevin Williams: Yeah, that, that, that I, if I never have to go into HubSpot or ClickUp again, I will be a happy clam.

but like leaves big platforms like Salesforce in a really bad place because ⁓ the is that they've been treating us. ⁓ We're them as headless. Like you don't need the UX, you just need the data structure. And the that seems to be forming up is we have a really good data structure.

And you and I have had the ClickUp conversation before. You're going to be using a heck of a lot more tokens than if you're using Sonnet 4.6, for example. So sort of setting the stage there.

I've said in previous episodes that it can be a little bit of a corollary. If you're looking at an investment or you're a CFO to figure out how well, or how at least much an organization is levering AI is to simply look at their general ledger and see how much they're spending on tokens. If they're at that $200 ahead. where I tried to create my own project management tool using some open source stuff and realized that the LLM didn't have a great appreciation for the architecture of the open source solution.

And it had a very keen appreciation of how ClickUp works and its structure around project management. So it made sense for me to just move on with my day and use ClickUp as a headless environment, right? Well, ClickUp's paying attention to that. And if I'm using it from a headless function, I don't need 15 seats.

I could probably only have three, four or five seats that are populating that. And that's an existential crisis for them. So of course their move should be, or will be, they're going to toll the way that I call on their headless database. But, but, but, but substitutes like, and switching costs.

A few years ago, it would have been laughable. That's not entirely unreasonable. If they have five people in their dev department and they're all in the $25 plan, there's just no way that they're actually even scratching the surface of this. And it scales up from there.

Um, you know, I think it could be helpful to develop like a heuristic for if you have this many number of devs, how much should you expect from a budgetary perspective to be spending on additional tokens? Token maxing is taking that to the extreme. to go into an operating organization and say, yeah, let's just casually move our CRM to something else. It would have been like, you know, a year long like deal and all of this.

Well. So there you go. There's the sort of the background of what we're talking about here. Elijah Szasz: Yeah, it's a good frame and it's probably worth mentioning that this is largely an enterprise issue, right?

These are developers in big companies that are getting this massive allotment of tokens. some cases, ⁓ to burn them as part of their job description, expected to be using these tokens. In cases, leaning on it way too hard and all of a sudden the allotment is gone and the budget is blown. in extreme case, Wong on the All In Podcast, Running it running it running a pretty small agency and having switched CRMs I don't know maybe five times over the course of 15 years.

It's awful every single time ⁓ awful. It takes it takes five times longer than you anticipate it will there are tons of migration issues There is the the re-skilling of what this new monster ⁓ capable of and how to find things in it It's not an easy lift. Yeah, there's a pain of change. Yeah Kevin Williams: Awesome.

Elijah Szasz: He said that if a $500,000 a year engineer doesn't burn $250,000 of tokens on top of that, that he quote unquote is going to be deeply alarmed. So kind of sets the stage of the breadth of all this. And I think then there's it's, it's worth mentioning, like, why is this happening right now? Right.

And is going to get a little bit into the conversation of agents, which there's just been. Kevin Williams: It's not an easy lift, but that's what they're counting on. They're counting on that being a barrier, but they can't be headless and restrict your access to the data. ⁓ therefore extracting your data is easier than it's really ever been.

And then reconstituting that data in a new form. And I'm sort of air quoting easy because this isn't a light lift. ⁓ as long as it feels like it's still a good bargain to use these platforms as headless, ⁓ don't see people shifting. Elijah Szasz: so much happening with agents right now and the tools and just the sprawl of agents right now.

if you continuously run these agents or you spawn sub agents that are looping, where an SA ⁓ used tokens in the afternoon, an engineer can blow through ⁓ millions while everybody's asleep, not even while working because these agents will just keep going all night. Kevin Williams: But if they turn up that agentometer and are trying to recover basically pro rata, the value that they had previously from per customers, which of course is what the market is going to tell them to do, increases the incentives of the market to find easy ways to create new things.

So a few weeks ago, we talked about a GitHub project that is a Salesforce clone that at last ⁓ has been starred 30,000 times. Is it as good as Salesforce? Probably not. Elijah Szasz: So that's where things get really interesting.

So I don't know, Kevin, when you're deep in a cloud code session, are you tracking your token spend or are you just letting it rip? Where do you fall on the spectrum? Kevin Williams: But if it has the right data structures and the LLMs can read it and that can be your headless source, then this is a problem for them. And I get that their BDIs are focused on the right things for their shareholders, but we have more options as consumers.

And that optionality is going to matter as far as how SaaS is going to play out, as far as how tokens are going to play out. And unless there's an absolute value on tokens, I mean, I'm a mere mortal here that it's pretty rare for me to deploy an agent that's going to burn tokens that sit outside of my Claude Max plan. And when it does, generally it's sort of on the edges. And part of that is because, well, I'm pretty good at being a, you know, air quotes, vibe coder.

What I'm not is an enterprise AI augmented developer. there's going to be opportunities for substitutes. Elijah Szasz: Right. And it's also worth noting that we're looking at a snapshot of a very, very quickly moving timeline.

Right. So when you look at one of these open source CRMs that you were assessing, like, ⁓ is it as good as HubSpot or Salesforce? No, well, not right now. Right.

I mean, it's this, this stuff is moving so quickly and the connectors are getting so much better and so much more stable week over week that it might be a completely different conversation three months from. Kevin Williams: who can launch 10 different independent agents that are building an enterprise software solution. very proud of the things I've built. They work great.

They're pretty cost effective to build, but they don't require me to spend tens of millions of tokens to make them happen. ⁓ I could have done it that way and it would have happened faster, ⁓ I don't think that me as a product designer, I have the capability to deal with that level of output and that level of complexity. Elijah Szasz: All Right. It's not this is this is not static by any means.

We're talking about today. Not not even like next month. So in a really public figure, like Jensen wants engineers burning half their salary in tokens. you think that that is just reaching for headlines, pushing Nvidia chips?

⁓ Or do see this being the Is this a sign of things to come where you have these quotas to be hitting? Kevin Williams: It is frustrating though, when I think about that foundation that I was working with, know, their foundation, they don't have like a ton of free cash. So this has to work for them. ⁓ is not-for-profit pricing through Anthropic and et cetera, et cetera.

⁓ ⁓ we, and they spend the next six months developing all of these automations and then the switch clicks, like now they're gonna be in a world of hurt. Elijah Szasz: Right. Kevin Williams: Again, I think it's really ego driven. This is very, very hard.

If you're looking at more of our clients or like mid market clients and they have a CFO and the CFO is trying to budget. And when you have all of these big variable costs that swing this way and that way, that becomes a real problem to figure out how to plan for this stuff. And most organizations are not going to be able to figure that Elijah Szasz: Yeah. Kevin Williams: ⁓ as far as the overall costing of this, and I don't think it's, truly don't think in their case in particular, it's, it's going to be that dramatic.

Otherwise I wouldn't have recommended that, that they lean into it to this degree. yeah, there, there is an arms race ⁓ or customer acquisition race going on with all of the big platforms. So who can centralize it the most? and GPT has really come out swinging in the last week with the new features.

So having some hotshot developer who thinks it's like an ego thing to blow through a bunch of tokens. But I'm not, I don't think outside of the massive companies with massive budgets, you're going to see a ton of that. ⁓ to be much more output focused as far as the dollars, you know, you and I go back and forth on this as far as commoditization of the models. And I think over time, more of this is going to normalize.

Elijah Szasz: Oh man. I know. I know. And I, it's, know, I intentionally just took my eyes off of open AI for a little bit so I could just go in deep into entropic and Google's offering and lo and behold, like, Oh, now it's the best image generation ever with really small text.

And you know, I'm seeing these side-by-side comparisons with nano banana pro. And then all of sudden I'm getting image gen FOMO. Kevin Williams: And yeah, the, the, the hotshot developers will want to always be using the highest end model, but in a lot of organizations, be using, ⁓ know, what will be a dim memory of Opus 4.7 to do things and it will end up being cheaper.

yeah, but, but the, the, the tokenization is going to continue. it's going say the tokenization will continue until morale improves. Elijah Szasz: And it really becomes difficult to pick a single horse in this race. And we've touched on this before.

And think that you really led with, well, it really depends on the existing ecosystem of your organization. Like that will often lead you in one direction versus another because everybody's making amazing stuff right now. And every other week somebody else comes out with something more amazing than the other lab. Things are changing.

Things are changing. Yeah. Kevin Williams: things are changing and being able to launch not just sort of a single point development solution, but really doing a really good plan. when we talk about, I think vibe coding as a term is probably going to go away.

⁓ when we talk about this more accessible, like natural language type coding approach, vibe coding, often it's linear as far ⁓ as Elijah Szasz: You know, I would say of the three leaders, Google is definitely more developer focused these days. Like, I feel like all of the new innovative stuff they're pumping out is really ⁓ developers. And maybe that's because Anthropic has done so well in that, ⁓ that, in that world and with enterprise that they're like, ⁓ maybe this is what we lean into more.

But yeah, how, are you addressing that Kevin, like with your clients that you work with? Kevin Williams: kind of going through the process and I won't lie, most of my processes are linear too. But if you're setting a really good plan and that's really probably the most important part of IBCoding is having a really thorough plan and that plan has identifiable phases, Code will already identify whether or not it should be dispatching individual agents to take those on. ⁓ it's frustrating because we're still, they, are coming together in such a way that eventually it's just not going to matter.

They're all going to have the connectors. They're all going to have a lot of context. what matters is where that context is coming from and the density of it. but at the moment, even though like Claude design, which we talked about in the last show is absolutely amazing.

Claude browser still has no image generation capability. So. that's going to only increase. And then what will happen is your output will end up being much faster at the end of the day.

Is that token maxing? I think it's just more efficient coding design at the end of the day. Token maxing is not gonna go away, but I don't necessarily think it's a thing. If you are in a role where image generation capability matters, at the very least you're going to be using a third party tool like ideagram or something like that.

But if you're serious about it, like if you're a, if you're a marketer and you want to create, just say a basic workflow of, Facebook ads would be a good example. Like you're running ads and you, you, need to create a base image. you can create the concept in Claude, but you need to go to either nano banana or a ⁓ GPT. Elijah Szasz: Well, it's hard to separate out this concept of token, token, maxing and agents as a whole, because a lot of these token furnaces that started popping up, came online around the same time as the open source project open claw.

And people would take this agent that could do all these crazy things that none of the frontier labs seem to be able to publish as a feature and hook it up. Kevin Williams: in order to make the art that you want to make. And yes, there are plenty of third-party tools out there that you want to do this, but I don't love subscription creeps. So trying to do it with the tools you already have is often better, but the gentrification of GPT much more powerful at the moment than Google.

can argue, Google does a lot of things, they have Opal, like, et cetera, ⁓ but... Elijah Szasz: to one of the models of the Frontier Labs that was on a monthly paid plan, like your Claude Maxxer Pro plan, and just let it rip and have this thing work all night. then Anthropic shut that door, like, ⁓ sorry, you have to use the API, you're gonna burn tokens. And that's when ⁓ I started hearing some astronomical monthly bills creeping up because it really hard to predict Yeah.

Kevin Williams: If I take my single image that I've created in image 2.0, which is GPT's new model, I can create a very easy automation that then changes that to all of the ratios you need. And this is annoying, but as a marketer, you often need 10 different ratios for different ad placements. And there are tools that do this, but generally they crush it, they skew it, they crop it, they do this sort of stuff.

It takes a bunch of time. Elijah Szasz: exactly what these things are going to use when at 11 PM you say, Hey, go build this and don't stop until you're done. Right. And it could just spawn a dozen sub agents and it will just go to town and it may or may not actually solve the problem that you're trying to solve or build the thing that you want.

But in that process, it is going to rip through a ton of tokens. So this whole idea of agents really coming online in a usable way. Right. Or just takes a bunch of time.

It's so fiddly. It just takes a bunch of time, right? Even if you're using something that's really easy to use off the shelf like Canva, still creating these new boards and uploading the image and then changing the aspect ratio and then making sure everything is centered. It's like, the little 10 minute jobs that really start to eat into your entire day.

⁓ And then this token maxing, which is either building things or using agents to build things, pretty difficult to separate. And they just seem to both be on the rise in tandem. Kevin Williams: Mm-hmm. So two points.

One, can we please start a venture studio called Token Furnace and raise a bunch of money? ⁓ they're super painful. And now I've built a little flow that does it not only takes the single image and turns it into 10 images, ⁓ I'm very particular about like file naming structures. So then it does all the file naming for me with the sizes.

And then it moves it to a folder on my desktop, which then I can grab it and just easily add it to Facebook ads ⁓ if that's I'm doing on that day. But I can't do that with cloud. Elijah Szasz: Hiring Vibe Coders only. You have a hundred million token requirement per night.

Kevin Williams: to blow through capital. Tokenfurnace.com. I'm going to buy this one this afternoon.

Sounds excellent to me. ⁓ other point is, the model been figured out yet. It ⁓ can't end being based just on tokens. That doesn't really make sense, ⁓ and it translate into business needs.

And there's a very valid argument to be made. Elijah Szasz: Yeah, yeah. Kevin Williams: Like I could bounce it off of some like other third party process via API and I could kind of duct tape those pieces together. Um, but frankly, the results probably wouldn't be as good.

I'd be paying for an additional API. I'd be paying for complexity that then I have to maintain. Whereas just having it in the GPT universe, like that's going to work, but I am paying for a Claude max. Plus I'm paying for a whole set of team licenses.

that you were replacing certain business functions that cost you money right now, not necessarily getting into the, I'm going to remove my accountant ⁓ then replace them with a tokenized version of the accountant, ⁓ was an economic value that that accountant ⁓ was either being paid ⁓ producing on the other side. And Plus I'm paying for Gemini ⁓ as far as experimentation because I still need them all. ⁓ I need video, okay, so Sora is dead from GPT, RIP Sora. ⁓ ⁓ can't let go of Notebook LM and.

⁓ Elijah Szasz: ⁓ man. Kevin Williams: wouldn't be the least bit surprised as these more fine tuned and subject model applications start coming out. That once it is really vertically integrated and once it is really baked in a very good way, there is a plausible argument that can be made that, again, I'm not saying replace the accountants, but if your accountant is costing you $10,000 a month, now the accountant isn't there, there's $10,000 of available funding. Elijah Szasz: You can't let go of Notebook LM.

Why can't you? I can't. I got way too much gold in Notebook LM. It's just too useful of a tool.

Kevin Williams: And if you want to develop videos, then like V VO three through Google, which you're now going to be paying for it costs, it costs about a dollar per eight second section segment at the moment to be able to produce those videos, but they're amazing on top of your monthly plan. that's they they've Elijah Szasz: Right. Kevin Williams: for solving that same problem. you're never, nobody's going to replace the accountant for $10,000 a month in tokens.

That's ridiculous. But $1,000 a month, if you have a really codified function that is being solved really cleanly and well, or $500 a month, I don't think that's totally unreasonable. as it's built right now, we sort of choose your own adventure and build this thing and that thing and whatever. You're not just pulling.

Elijah Szasz: Are you talking about on top of your monthly plan? There's no, there's some video you can generate that's, but it's very, very little, right? Yeah. Kevin Williams: I think you can get one or something like that.

Yeah. And you can't use the latest and greatest model. ⁓ yeah. So it, ⁓ it, we're still in this place where you, you, you may have to pick and choose.

My advice is still lean into one platform, lean into the platform that people are using and offers the most connectors ⁓ most people's cases, that's going to be GPT or clock. And the fact that that now connects to Microsoft is absolutely great. and then if you have special needs for, digital accountant off the shelf and plugging them into your system, it takes a ton of other work to make all of that stuff happen if you can make it work at all. And that's harder to justify the value.

⁓ something else like video, then maintain one subscription for Google if you need to. But you and I live and breathe this, like this is my profession, so of course I have all of the subscriptions. Elijah Szasz: Yeah, it's, feel like it's like almost everything else we talk about where you see these instances or these stories of these things happening and think, is this future? Is this the way it's going to be?

And the answer as always is maybe, I don't know. Maybe it will be. mean, you know, the darker read might be, ⁓ you know, if token spend per employee approaches ⁓ that employee's salary, headcount math starts to look Right. Well, keeping in theme with the token furnace and agentic use, is like, yes, agents, they're the future for developers, not for all you normies out there yet.

Anthropix we'll give you co-work. It's kind of agentic if you don't want to just build something from scratch yourself. I've gone super deep in co-work. ⁓ And ⁓ interesting.

days I'm really different, right? Like if compute's doing the work, the question then becomes many humans are needed to coordinate that work ⁓ that whole ⁓ becomes a little more difficult to avoid, right? So the token budgets are being framed as a perk. They actually be the audit trail of what humans end up being ⁓ needed that fallout.

So, right? mean, it's ⁓ easy imagine these different directions of how that's going to go. very, very impressed with what it can do with chaining tasks together. Other days, it breaks and it can't explain why, and all the lights go out, right?

Kevin, ⁓ tell New just... wow. Get one out of there. New capabilities in GPT for agents that anybody can allegedly just spin up themselves and start using.

and trying to separate that out from, you know, Jensen Wong trying to grab headlines. you know, so ⁓ connection to the agents, you know, agents, I feel like every week like ⁓ the whole of AI is new again. But I feel like this ⁓ this last there was just so much going around about ⁓ agentic AI. I think one of the big stories that came out was Where are you in this?

Have you ⁓ around with it? Because just mentioned with the image generation, like, ⁓ I can now say after you the image, put it in these different aspect ratios or do this different color treatment or whatever else you want to do that you'd have to use a different art program for historically. Now, when you're about is that with the new agentic tools that they've layered inside of the subscriptions? OK.

Kevin Williams: So. Elijah Szasz: Copilot which I feel like has always been the redheaded stepchild of I told that we've ever ⁓ about ⁓ has this this ⁓ agentic capability in these products that Tons and tons of people use like word and Excel and PowerPoint, know, not not not agent mode is like a separate tab anymore It's like the default copilot experience Kevin Williams: Yes. ⁓ so in your GPT on the left bar, sort of depending, but it should say agents and you click on agents and there's a, it's sort of a little bit hard to find, but at the top there's like a gallery or featured agents and you can scroll through and you can see all of them.

from a template, they're basically templates. from a form factor perfect perspective, this is great. This is way, way, way better than cloud cowork. as far as don't get me wrong.

Elijah Szasz: And there's templates there too, right? So you can take these multi-step actions directly inside the documents, stuff that would just ordinarily take a lot of time and pixel pushing, like formatting and restructuring, building visuals, transforming data, all of that you can just do with prompting right now. So I'm curious, since you're much more plugged into these traditional corporate environments than I am, are you starting to see this? Are people adopting it?

Have you played around with it yourself? Kevin Williams: Still love you Anthropic, like it's such a, it's, it's sort of a, it's a complicated environment to deal with coworker. Whereas open AI has taken a really clean, straightforward pick and choose, get a template and go with it. But they've also offered centralized templates.

So if you're on a team's plan, you can one person can create the template kind of like custom GPTs. now that. Um, so I just did a 90 minute deep dive with, uh, with a really forward thinking, not-for-profit yesterday, a foundation, um, and it was just like fully leaning into this, which is, which is great to see, but I feel like I hit them with an information cannon. I feel slightly bad about it, but, um, what I'm seeing is that nobody cares about co-pilot.

Um, they want co-pilot to work. Uh, what's also happened, and this is a great tip for our audience. Elijah Szasz: I was just going to say back in the custom GPT days, like that was the big boon, right? That you could create one of these, share it with their whole team.

You could have somebody with publishing rights who could then update it for their whole team if something changed or the workflow had new demands. Okay. Kevin Williams: Yeah, and it works. It has tight admin controls as far as centralized control.

somebody the organization with admin rights has to give it access to the different connectors, which I think is totally fair and totally what you want. But once those are enabled, then it is pretty click, click easy. And there's something about it that's just a little bit more user friendly. is Microsoft has opened the gates for you to add GPT or Claude directly into your Microsoft 365 app.

And they're basically almost admitting defeat in that people prefer to use the interfaces that they prefer to use and forcing them to use Copilot hasn't been super productive. Copilot is still super limited. just here's a test for the audience guys, like add ⁓ I personally don't find Claude desktop to be particularly user friendly. It's designed to do too many things for too many people.

And as a result, it becomes a little bit overwhelming for kind of normies who are trying to deal with it. ⁓ we're clod or GPT to Word, to Excel, and to PowerPoint. And it will just, you just click on the little add-ons button at the top and all of a sudden you'll have a clod button that's sitting there. And because it's clod, it's also connected to through an agentic harness, I'm air quoting, that's a big topic that we need to cover sort of in depth at some point, that Elijah Szasz: It's, it's, it's very convoluted.

mean, it's, it's now gotten to the stage two where there will be a button in the settings that you click to, for example, you know, see your connectors or browse connectors. it says, ⁓ this no longer lives here. ⁓ go back here. Like this even redirect you to the right place of the app.

Like it really does feel like, I mean, they are shipping stuff at a breakneck pace. And I'm sure that's why the UI feels like it's starting to fall apart, but yeah, it's not so easy to use. So you, you find it a much more, Kevin Williams: your cloud is already connected to your CRM and your project management and your email and all of these other things. And now straight from Word, you can use those connectors to basically pull in the data that you need and then create the document that you want from that.

So that alone is amazingly powerful and would be very, very hard to do in the Microsoft only environment. Then actually link the documents. Elijah Szasz: the amenable user experience inside of GPT right now to get to these tools. Kevin Williams: Yeah.

it's, it's straightforward. Have I gone super deep on it? ⁓ ⁓ I'll, I'll admit. So we've talked endlessly about our cowork like chiefs of staff.

⁓ that is what I, where I started, I tried to replicate a lot of the things that I was doing with my Claude chief of staff ⁓ ⁓ sort of the edge ⁓ kind of got me as far as, I need to connect to this custom data set or that custom data set. And it was relatively easy to do in Claude cowork. So if you have a ⁓ Excel document that's open next to the Word document, you can actually reference, hey, look for the open Word document where we have all of this information that I pulled in from the CRM.

Now populate the spreadsheet from that information and it will pull across the open browser tabs, not even browser tabs, you're at like app tabs that you have. And now if you've done the same thing with PowerPoint, Elijah Szasz: Okay. Kevin Williams: and maybe a little bit more finicky to do in GPT and you can say, link together the spreadsheet document and the Word document and create the follow-up document. It is really cool.

And it actually works in a way that's really pretty intuitive. that, that, that, agentic is one key. Having the actual workflow tools like the productivity suite of Microsoft is another key. ⁓ And being able to.

Elijah Szasz: Is that because he like off the shelf connectors already existed in cowork or that MCPs just worked differently in GPT? Kevin Williams: Well, I think it's because I'd already done the lift and that in the Claude universe, I'd already basically established the pattern to connect to like my aura ring and you know, other things that I've built and the knowledge based in the context of Claude was able to pull that because I built it in Claude. Whereas didn't really have the context.

And then I ended up trying to go between Claude and GPT. And to be honest, I simply ran out of time. ⁓ I'm, I'm. like seamlessly meld amongst them is, is really, really neat.

And I think that people are going to get a lot out of that. And then I haven't been able to unlock it yet because frankly, I don't have the right level of Microsoft account, but there's also the ability to create cowork. So Claude cowork type tasks right in Microsoft. we'll, we'll talk more about that as that, that matures a little bit, ⁓ I'm it's a, project for this week to get deeper into, little bit and see how much better it is, but sort of results TBD.

Elijah Szasz: Yeah, did you get any sense as to how far you can run with this on one of these paid plans? So for example, if you are on a $20 a month plan with Anthropic and you start using cowork, you're not going to get very far. And by not get very far, I you'll get like the first half a dozen steps of what it is you're trying to accomplish. You're like, sorry, your session is already up and you got to bump up to a hundred bucks a month.

⁓ When you say, when you say right inside of Microsoft, like inside its suite of different products, like access it right inside of Excel or word or anything else. Kevin Williams: Yes, right inside the suite. And you know, the other thing that Claude has released and this stuff is, this stuff is dropping so fast that like I'll be mid demo and I'll see something new. That's like popped up, which is exhausting.

and yesterday it was that ⁓ a Claude paid teams account. you have to have five people in a Claude teams account. They've now added. Elijah Szasz: Yeah.

I'm so tired. What does that look like? mean, because there's a lot of stuff bouncing around there as far as, you know, back to this conversation of what's being subsidized, say like, open AI is ⁓ subsidizing a lot more than the other labs, right? Have you seen that to be true with this experience with ⁓ the agentic Because that's where you really start to see the dial of usage get turned up high.

Okay. Kevin Williams: ⁓ a layer of enterprise documentation such that you can have like unified context amongst all five or 10 or 50 of those people as far as what your organization does. And since it's unified in Claude, it actually becomes unified in Microsoft as well. So when you pop open a Microsoft document, if you've done a good job of that on the Claude side, theoretically, you should be able to populate a brand aligned Word document as well.

Oh, a hundred percent. Like absolutely a hundred percent. Um, that way more for my money. And I'm also doing experimentation with Codex, which is a GPT's version of, um, of coding of coding.

And, um, I, I probably shouldn't be doing this during a podcast, but on another screen I actually have, uh, um, shouldn't be doing it not because it's intrusive, but it's actually kind of computer intensive. I'm running a pretty elaborate Codex project. Elijah Szasz: Like you're getting, you're getting way more of your money right now with GPT, with the genetic use. Okay.

Kevin Williams: by pulling through the Claude knowledge and pushing it into your document. Elijah Szasz: Cloud code. That is pretty insane. What is the level of effort of actually setting that up or the connectors doing all the heavy lifting?

Kevin Williams: So, okay, I laugh again. I, I, I, this was a great group, but they were, I, there wasn't enough time. I needed like a week to go into like all of the bits. it's there are admin controls.

So you need to have your admin controls in place such that you dictate what people can connect to. I highly recommend that if you're doing this and to serve getting your toes wet, you really only allow read. Elijah Szasz: lashing our podcast. Kevin Williams: And it's been running for a long time.

It's been running for like two days as I sort of drift over there and I just sort of click go and I'm kind of having it run with a concept I wanted to do. This is a codex build. ⁓ ⁓ I did my planning ⁓ what I was going to do in Claude because Claude is still generally accepted as being better at overall product or design planning. Elijah Szasz: Okay.

This is a Codex build that you've had going for two days. Okay. Kevin Williams: basis instead of a right basis. So as in it can go out to these connectors, it can pull information from OneDrive or SharePoint or Outlook or wherever it is, but it can't on the anthropic side.

So you have to, you can set up the connectors from the admin perspective. And then it is a little kludgy because individuals then have to basically themselves click the, again, there's a little button that looks like four squares. built a really detailed spec there. Then I moved over to Claude design and had it do the UX design.

And then I basically merged all of that, dumped it into Codex and walked away. And we're going to see like what comes out the other end of it with really good planning, really good UX and sort of blind like click through in Codex. I have the point being I haven't run out of tokens. ⁓ hasn't even like threatened me or menaced me.

⁓ Elijah Szasz: Are you talking about on the anthropic side or the microsoft side right now? Okay Kevin Williams: the top of your Microsoft apps and it's add-ons and you just search for Claude or you search for GPT. ⁓ works a little bit differently in GPT. You log in from there and if your organization has granted the access to it, then pretty click, click, click easy.

But in people in this idea ⁓ of having ubiquitous Claude. So, okay, now we have Claude in the browser. Elijah Szasz: Well, I was going to say, are you playing the Alucard token game like you do with Anthropic with OpenAI of it a little bit more? Here's another 20 bucks, here's another 100 bucks.

Are you doing the same thing over there or are you just on a flat plan? Kevin Williams: I'm just on a flat, you know, $25 teams plan. And, um, I, it's been a long time. Okay.

A it's been a long time since I, I, not a long time, a couple of months since I was like a power GPT user. I would be using GPT for like eight hours a day. Um, and I don't remember ever running into a limit in the way that I do in Claude. Um, there are stability issues with Claude, like there are cost issues with it, et cetera.

Elijah Szasz: Okay. Kevin Williams: Okay, we get that. We can download Claude and have it locally in terms of Claude desktop. And that's what we've been talking about in prior weeks, as far as running like local agent flows and things like that.

You can now have Claude in your Microsoft apps just like baked in there. ⁓ you can have Claude in your browser. So ⁓ that's doing things sort of on your behalf. Elijah Szasz: Yeah.

A lot of stability issues lately. We've had a lot of text exchanges over the last week of, going ⁓ ⁓ it down for you? I'm like, ⁓ no, just, ⁓ yeah, co-work's down for me. ⁓ yeah, WebUI is down.

Yeah. Yeah. Kevin Williams: So there are four different like obvious places where it appears, if not five, if you count your phone and having like continuity between all of this, that's the direction that's heading. And I just heard a snippet from in podcast ⁓ on of the end of prompting.

And I think they got it a little bit wrong because ⁓ you me, I'm really serious about prompting and we teach like crit prompting and whatever, and whatever. is actually really important. Yeah. So I'm not anti-GPT and their new model, their 5.

5 model, which is the one they just dropped is a little bit better at writing than it was previously. If you are already baked into it and a lot of people are that, you know, the thrust, me of this time last year, I would have been, yeah, a hundred percent, just lean into GPT, get a team's plan, ⁓ your people, teach them how to do projects and custom GPTs, teach them how to prompt, and you're at least off first base at that point. But CRIT, which context is the C, becomes less important as the LLM knows more about you.

So when you have a little baby clod and it doesn't know anything about you, prompting is super important. But when you have like the adult clod that's connected to everything and can see everything, and that's a little scary, you don't have to prompt the context as much because it has all of that context. So. Then the FOMO kicked in with the media cycle, you know, that's included over the last, ⁓ you know, two, three months of Claude Claude Claude is amazing and Claude is amazing, but it is, I think a little bit more of an advanced tool in some ways.

if you're, ⁓ more expensive, and if you're already in culture aided in the GPT universe as of this week, they really have caught up. So don't feel that sense of FOMO, lean into the platform you have. Elijah Szasz: Yeah. more expensive right now.

More expensive than the end user. Kevin Williams: ⁓ It gets easier and easier and easier to use and theoretically we're going to get to more of an anticipatory point where it knows before you know that you have some sort of a calendar conflict or something like that and that proactivity is going to start popping up in workflows probably in the next few weeks. because the other challenge you have is the baby LLM problem, which is if you do switch to cloud, yes, there are some approaches you can do to like enculturated into your way and the way you speak, but there's value in the 20 something thousand chats that I have in actually have 30,000 chats in GPT and probably 18,000 chats and Claude at this point.

⁓ Elijah Szasz: Yeah, we've discussed this. mean, this is reminiscent of the mono thread that we talk about, right? Of having a single point of access where instead of diving into half a dozen apps on my desktop or my mobile phone, I have one conversation and it's got time context for everything that's going on between my calendar, different email accounts, project boards, whatever it is, a GitHub repo could be, it could be anything. And based on what's on my calendar, what's coming into my inbox, where I am in my day, it will then Wow.

What is it? Yeah. So I'm really curious because that's a lot of chats. ⁓ Is that because of historical conversations only before making the switch or diversifying across different models?

Or do you still have very specific tasks or workflows that you choose one versus the other? For example, tested the models extensively with content creation, Kevin Williams: You Elijah Szasz: deliver what is most appropriate to me that is then going to be helpful without me going to a bunch of different endpoints to find that information. Like that's kind of the dream. That's the holy grail of a lot of these AI workflows.

thing I was wondering with ⁓ copilot rollout of the agentic use is that it's one more example of the just seems to be best in class for writing, right? And then at the top of this year, everyone's like, ⁓ ⁓ can touch Claude code for any sort of development tasks or planning, right? Where you in that divide now? If you said that OpenAI has ⁓ basically up at this point and that there is no buddy with a great lead, how are you forking ⁓ that?

costing didn't change, they rolled out these new features and it's agentic and we know the compute burn on that. So, you know, if Microsoft eats the compute cost forever or is there a per seat surcharge coming once, you people become dependent and this kind of goes into this whole notion of we're living in this AI subsidy era right now, right? Where workstream, like, what do you have, like, I only go to this model for this task and this one for this one, because I mean, geez, I feel like it was just a week ago.

You're like building a million things with cloud code. And now you're like, Codex ⁓ for two days. ⁓ Kevin Williams: I'm not, I'm not, I'm not defecting. I'm really not.

do like playing codecs off of Claude code, which is an interesting thing. Sort of like, like, your arch enemy Claude said this. How do you feel about that codecs? And they like sort of beat each other up and you can get a sense of what, the direction you should go.

I think that GPT still definitely lacks in design and UX. So I would struggle a little bit given the Elijah Szasz: there's no way that the utility that most people are getting, whether it's a $20, $100, $200 a month plan, or even arguably the cost of these tokens being piled into the furnace are actually equal to the amount of compute cost from the labs themselves. And when does that party end? And people might say that some signs are starting to come up of that.

⁓ I do that all the time. Yeah. Kevin Williams: sheer quantity of like dev-based output that our team is doing to shift entirely over to GPT, even though it would be more cost effective, ⁓ would still have the planning and the design layer in Claude. And this is sort of the rub that ⁓ can't yet really lean into ⁓ ⁓ ⁓ you have anything approximating an advanced use case, be it development or vibe coding or images or Elijah Szasz: party beginning to end with just changes in service and pricing.

Are you familiar with this millennial lifestyle subsidy? Have you heard about that before? Kevin Williams: No, but it sounds horrifying. We're gonna sound so old.

Elijah Szasz: You'll love this one. It sounds horrifying, right? Well, okay. So yeah, this is a little dated.

So Derek Thompson at the Atlantic coined this back in 2021. So here's the original framing. Like if you woke up on a Casper mattress then ⁓ into your basement and worked out on your Peloton and then you Uber to your WeWork and then you ordered Tardash for lunch, ⁓ took a Lyft home and then had Blue Apron for dinner. Kevin Williams: maybe you need to do video or maybe you need to do multimodal as far as sound.

Like you're still going to be having to pick sort of best in class and you have the luxury of doing that right now. So it's not, it's not a giant deal. Elijah Szasz: because it's being subsidized. Kevin Williams: It's yeah, because it's being said.

I mean, we'll, we'll see. just, I don't think I'm wrong about the commoditization that that's, that's the counterpoint to, to this. we're suddenly going to get charged. Everybody's going to have to get charged $5,000 a month for this.

⁓ don't buy it. I just don't, I don't buy that that is the extremity that it's going to go to. Elijah Szasz: You probably interacted with at least eight different unprofitable companies that collectively lost $15 billion in a year. So the whole idea is that VCs and zero rate money paid for the difference between what you paid and what things actually cost.

And that era ended mid 2022 when all the rates went up. So if you and your buddies Yeah, I'm so curious how that's gonna shake out. Kevin Williams: Well, I mean, hopefully, mean, that, but $5,000 a month versus having, you know, five employees is not actually a giant ask. So if they can make.

And really doing it and not just like creating more chaos and, know, more work and, and whatever. And they have not done a good job of establishing that value proposition yet, but that that's next. We were like out of time, but. Elijah Szasz: We're really stoked on taking an Uber black car out to the clubs every Friday and Saturday for the price of a taxi.

It was a rude awakening when all of a sudden that same ride was 150 or $200. So a lot of people are looking back at that and thinking, what's the parallel between what happened then and what's happening right now with all of this VC subsidized AI compute that everybody's getting? Right, if it's actually doing that work that the employees would be doing, right? Yeah.

Kevin Williams: but talking about like Mercore and Scale. So the piece of news from Facebook that they're basically doing key logging of all of their employees to that information to train the models, ⁓ to replace those same employees who seem to have some opinions about this. Elijah Szasz: ⁓ my goodness, yeah. ⁓ it comes to pricing or token or anything else, again, what are we used to?

Not of, I like Peloton experience, but more of this is now deeply embedded into my workflows. Right. Spyware on your machine, not to see if you're productive, but to take your productivity and create an agent out of it. Yeah.

Yeah. Kevin Williams: And to create an agent based on that's where they're heading. That is their, their Silicon Valley BDIs are on how can we do what's already happening to devs and apply it to accounts payable and apply it to HR intake and, and et cetera. And I'm positive that you're going to see that model come on the market of this is your HR intake.

Agent. And for a thousand or $2,000 a month, they are going to be able to do everything that that former human would be able to do. And some organizations are gonna bite on that and others are gonna balk at that. But a decent value proposition if it's presented that way.

⁓ that's different from this nebulous like token consumption question ⁓ the industry needs us all solving these problems such that they can mature to the point that they have some sort of stability in their like inference costs and et cetera. ⁓ cutting us off at the root. too early is going to be a problem for them long term. ask yourself if Claude was $5,000 a month, that would be a little steep for me.

But all of a sudden you'd find me playing with Keme2 or with DeepSeek and some of these other ⁓ running models. And ⁓ would me all of five minutes to decide to do that. And it would take me all of a day to have something functional that's up and running with that. Elijah Szasz: for sure running local models, yeah.

Kevin Williams: That's what they're up against and they'll try, believe me, they'll try, but I just don't think it's gonna be quite as clean cut as maybe you're intimating. Elijah Szasz: It's a weird time because on one hand it feels like this has just been going on forever. And I think the pace of it makes time just stretch out. But the reality is this is so early.

No idea where this is going to go. mean, got a new CEO who's really big on AI and that could change the whole game, right? And you've got all these Kevin Williams: Yeah. Elijah Szasz: different labs that you use for different workflows inside of a relatively small organization.

know, ⁓ in the day, it's like, ⁓ we are all Apple or we are all Microsoft or all Google workspace, right? And that's just not the reality right now. And so you wonder if it's going to lead into a time where that's what it feels like, where you kind of you pick your horse ⁓ you stay with it. And that becomes how your organization is run, like where all these tools become relatively equal.

So who knows? Yeah. Kevin Williams: All right, we're have to park that one, but I think it's important for you to know that tokenfurnace.ai is available.

So if we wanted to spend $100 a year on a joke, we could do that. Elijah Szasz: Mmm. Mmm. ⁓ Maybe it could be a leaderboard, like a corporate leaderboard.

Right before you install your spyware, look at keystrokes and train agents, right? ⁓ Kevin Williams: corporate leaderboard. so, hey, thank you. Thank you everyone for listening.

Please tell your friends, like, follow, subscribe as the kids say, and we'll talk to you next week. Elijah Szasz: you All that fun stuff. See ya.