Allie K. Miller: Find your "weirdos" - and let them lead

AI is not a tool. If you think of AI as a tool, you are going to budget as if it is SaaS or traditional procurement, roll it out as if it is a traditional SaaS platform. You're measure its impact solely based on productivity, and you're not going to think of brand new business lines. org restructuring.

process reinvention. None of that really happens if you're just rolling out a productivity tool. Welcome to a special live recording of Work Lab here from the copilot summit. I'm your host, Molly Wood.

Our guest today is Ali K Miller, the most influential voice in AI business. She is the founder and CEO of Open Machine, where she advises fortune 500 companies on their AI transformation. Ali, welcome to work Lab. Thank you.

Love being here. Live is really fun. okay. When we talked before this podcast, one of the things that you said to me and I want to start here is that and I feel like a lot of you can relate to this.

Is that a big part of your job is therapy. So let's start with therapy. What are you hearing and how are folks feeling? The emotion has changed a little bit in the last couple weeks.

So I advise CEOs and C-suite on AI and transformation. And it felt like the last about seven weeks, that another moment has shifted. And I think a lot of these companies, honestly, the leadership is just very terrified of making the wrong decision. There's a stat that 74% of CEOs are just worried that they're going to be fired in the next two years because they handled AI wrong, not that they adopted Twitter, just wrong in general.

And so I think a lot of these leaders are making sure that they understand the context that we're in today. We're kind of in this like third shift of generative AI. And I get the sense that most leaders are still operating under 2024, 2025 assumptions. We're going to talk more about that in a second.

Before we do that, who here thinks that they might be fired anytime in the next two years? If you raise your hand proudly and say. Yeah. Right.

I mean, we're sort of joking, but we're also not joking. I think that is is where we are. It sounds like my understanding from what you've talked about over the last couple of days here at Copilot summit, is that the sense is that the tools are there. The sense is that the strategy is starting to move in the right direction.

Is that a fair assessment? Kind of okay. The lack of hands shows me that they're all feeling what you're saying, which is that for folks who thought they had it down and were headed in a certain direction, there has recently been a shift. Tell me about that.

So I guess first moment was, you know, 2022, late 2022, and then into 2023, it was like general chat bot adoption. The kind of second shift was end of 2024 is with like big reasoning models and then the shift in the last couple weeks has been really in this, like multi-agent space. This idea that I can actually work autonomously on many tasks at a decent reliability level for an hour or more. And so a lot of companies like, I don't know if everyone else is freaked out about costs, but that is the left left.

Thinking, okay, I think that because. That is the number one thing that's coming up, which is just if you're in this multi-agent space, you're using frontier models. If you have people like me or like your engineers who are running amok 24 seven and not 9 to 5, then your costs, instead of being thousands per head per year, there are companies spending thousands per head per day, right? Mostly in the engineering space.

But that is a very real reality. And so a lot of these conversations, on the technical side at least, are just about orchestration, efficiency, and how they could maybe even stratify their workforce to say, okay, maybe this is our really big spend center. And as a whole company, here's where we're going to be adoption wise. But the big, resource allocation might go to like a frontier unit as opposed to a company of 80,000 people.

are these two things reconcilable, the multi-agent shift that we have seen where you do need to be automating things and letting them run overnight and keeping costs down. So the seven weeks are introducing a lot of variables here. Yes. in the last couple of years, basically every well, it used to be every year now it's every couple months and I swear it's going to feel like every couple of weeks we move between paradigm new thing coming out and then efficiency of that thing.

And we saw the same thing with cloud. Like this is a common thing to see. It's just that the pace of change is so much faster now. So that first big push of okay, these new models are coming out now.

We have brand new capabilities. We're just in that experimentation phase. We're starting to see some of these smaller models come out. Maybe they'll be some place for open source model to come in as well.

But we're we're in the thick of experimentation. This is actually a time to kind of lean in on costs, again, not for the whole company, but for maybe a group of 60 or 120, depending on how big your company is to be able to completely put their foot on the gas. You want those people running experiments so that you can figure out the brand new way, working to then bring it back and then impact the 80,000. But you don't you don't really want 80,000 people spending thousands of dollars a day while you're in the experimentation phase.

And I would say that this has been a bit of a tough tension, right? Because we have, I have now done many conversations on Work Lab about the idea that if you silo adoption to pilots, that can be a problem, that you really do want everybody in the organization and I'm sure you've heard this, too, to be involved on some level. And yet we're kind of in a state where it sounds like because of cost and and the newness of the capabilities you do want to. I like this idea of a frontier kind of type of thing.

There's like double pathing of company progress. So you have to take the 80,000, minus one, 20 or however big you decide to make this thing, you have to have all of that crew still getting amazing upskill opportunities, still experimenting within their departments, still learning, still going into their teams channels and sharing new things that they're working on. Town halls, jammies, all that stuff is happening in the 80,000 -120. In the 120, the second a new model comes out, I want them immediately testing that model.

They should have a higher budget than the rest of the company. They should be working in teams of 2 to 8 to be able to actually get production, prototypes done so that it can then roll out to a bigger department. but your, parallel path thing, whole company and frontier experimentation and taking the best of the frontier and bringing it back to the others. Where are companies sitting now?

I mean, you know, one of the things we've been talking about is that you said that this this kind of shift is happening just as people might have felt like they caught up a little bit with the first shift or the second shift. What does this really look like in practice? What are what are companies not doing now that they need to start doing tomorrow? I would say that they should.

See how long that list was in her mind. I'm sure we have 37 minutes. So like don't. They?

They feel the moment. And I think acknowledgment is very important. They feel this pressure. And honestly it might be because of the cost push where we didn't have that same pressure.

And in 2024 when the new reasoning models came out. So just felt more deeply at the C-suite level. I think the average C-suite has not built a single agent, and that has really, really bad implications for the rest of the business, right? Like the best AI strategies have both top down and bottom up, where you have your CEO and the C-suite all in on at least setting an AI vision, And I'm still getting pings from CEOs who kind of whispered to me in the hushed tone of, like, I haven't used any of this stuff like, can you help me?

in the last six months, I've done more C-suite, one on one individual workshops and tutoring and coaching than the last three and a half years. What about other management layer? So there was a, Microsoft study of 1800 employees globally, and it found that when managers actively model, I use themselves CEO all the way down, ideally. Right.

But at any level, employees reported 22 point left in critical thinking about their AI work and a 30 point lift in trust of a gigantic AI. The middle layer, the summary is, is where strategy lives or dies. I feel like there has been this kind of polarity in organizations where the strategy is being set, probably at the level in this room, with C-suite approval, and then employees have been told, you have to go and learn this, probably on your own, or you're not going to keep up.

And this middle layer thing feels really important to me. In addition to CEOs who have ever used one of these tools that feels basic, you know, but you never know. I mean, everyone is very busy, like, yeah, carving out intentional time to learn AI. And again, maybe they were giving themselves months to catch up, but now everything is in the span of weeks.

What I would say, and this is another leadership gap, is that the majority of leaders that I talk with, and this goes back to the cost question as well. They're prioritizing productivity. They're prioritizing just a higher amount of output at the same cost, maybe slightly increased costs, hopefully decrease costs at some point. But it is a full productivity driven conversation.

And like, I would love to say that the average employee is so freaking motivated to just hundred bucks their output. Absolutely not. They love their KPI value. Not.

It's not encouraging. It's not motivating. It's not empowering. Like no one wants to feel like a just a workhorse, a drone.

And so you have these C-suite that are very motivated for showing growth for shareholders. And so they are going to be motivated by cost reduction and output increase. And then you have employees who are, to be completely frank, very fearful for their jobs. They have other blockers.

Maybe they haven't gone through upskilling or their company taught them AI three years ago. And they said good luck and they have not kept it up And leadership is just throwing these horrific numbers at people, and assuming that the motivation follows does not exist at all. This is a people problem. I know all of you are running phenomenal AI strategies, but tech is a percent of it.

It is less than 50% of your strategy and figuring out how to make that culture shift working with your CRO, working with people who motivate the rest of the company like that is what is underappreciated. we say that strategy lives at the middle layer. Adoption lives and dies at the employee in the cubicle. I think adoption rates are one thing to look at.

And I hope all of you know, your monthly active, weekly active daily active I users inside your org. But what I would love people to look at is what are they actually using AI for? Because if my whole company is using AI to rewrite emails and you put it in and you say, can you make it three sentences and make a nicer like that is not meaningful AI adoption. you might have people who believe that they're amazing AI users and who are using AI every single day, and they're using it at 5% of the power, can I give you just want to say please.

Okay. I was in a board meeting and the CMO of a massive company. He raised his hand. He goes, you know, we don't use AI for writing because it's not good enough.

And I was like, what do you mean by not good enough? And he was like, I just feel like it doesn't have heart. It doesn't have the umph. And I asked, I was like, well, how are you using it?

And his prompt was like, well, you know, I asked it to be funny. I asked it to be heartfelt and it wasn't. And I was like, did you iterate back and forth through that 50 times? He said no.

And so there's these like, this is an insane term, but just go with it for one second. We call it like Gremlin mode at our company. So mode one is what this guy did. You asked for it.

You give it a little bit of description, you get it back, you're disappointed. You think that AI is horrible, you think everything is overhyped. You don't invest in it. You're not.

The level above that is like I gave it. Examples of a blog post that I wanted to write. I iterated back and forth of it. I would still say that is 10% level.

The Gremlin mode version of that that I told this man, and I think you could see as the color drained from his face, that he realized that he was stuck in 2024, was that let's say that you give it ten blog posts. let's say that you're, I don't know anyone want to shout out their company. Name. Or industry industry?

Give us an industry health care. Okay, so let's say that you're in the healthcare space. You're writing ten blog posts. You look at your top performing blog posts, you take all ten.

You put all ten individually into AI. You have. I write a briefing to be able to write the, code blog post. You then take that brief, open up a brand new thread, have it write the blog post based on that, and then you iterate with that.

You literally have it loop eight times to be able to improve that blog post. You give it your final blog post to say this is actually the perfect version, and then when you come to it and you want to talk about, I don't know, hospital recidivism rates or something, you then say, great, write a new brief. But that process was six hours and you're using AI is goal features and looping features to be able to get to that? This is not we're not in this like single chat thread land where you're giving it one sentence that is not at all what these systems look like today.

Okay. You are alluding to effectively training, right? And it's so funny because I feel like I know all of these people and I say, oh, I trained a model to do this. Now we are training agents to do things.

It sounds more intimidating than it is when what you're really describing is sort of an upfront investment of time in making a tool work well, which everyone can relate to. How much more complicated does that get in a multi-agent world? the biggest thing that I tell all of my clients AI is not a tool. Like if you think of AI as a tool, you are going to budget as if it is SaaS or traditional procurement, you are going to roll it out as if it is a traditional SaaS platform, you're going to measure its impact again solely based on productivity, and you're not going to think of brand new business lines.

You are not going to think of org restructuring. You're not going to think of process reinvention. None of that really happens if you're just rolling out a productivity tool. I think the way that we're working, there's a weird skill set and yes, it's upskilling on the actual tool, but it's more upskilling on the mindset.

There's like weirdness. And the three skills that I look for - and by the way, if you have not updated the way that you interview people in the last year, you are very behind. The way that I am interviewing, the skill sets that I'm looking for, it's three things, and this is what I think makes people succeed in the AI age. Number one: Do they have high agency?

f they are that CMO that had that problem, are they going to let the AI dictate their day? And if you didn't get what you want, are you just going to walk away and complain and still do things manually? Or are you going to go in bulldozer mode and try it from 17 different angles? So high agency, strong sense of wonder, and this has nothing to do with age, nothing to do with department, like you have - Some people are two years from retirement who are the most curious people I've ever met.

And then the third is systems thinking. And that's been the really big change the last year, where we're managing multi-agent systems. Like I have 34 agents for me working backstage, that is happening and I am managing those 34 right now. And the average person that you might interview, that's at the IC level, who has never been a people manager, might actually have a pretty difficult time managing a mini org, a temporary digital workforce of AI agents, because they just haven't been exposed to that.

Early in career people in a really tough position on that systems thinking side, very strong on the AI side, but that systems thinking side, if you've been a people man, and I know all of you are, you have an unbelievable strength going into a multi-agent world. We are rolling out full digital workforces and trying to make that consistent across all employees and all that. But systems thinking that is that number one big shift. What should we call this investment layer?

Just thinking about how -Running a business. in 2026. -it's not... Yeah okay, but it needs a cool name.

-Okay. -Everything needs a cool name. But really, like, if you were going to put this in your PNL, and you were going to say, “I need to differentiate this from SaaS. I’m working on my tax returns for next year.

” Is it headcount? Is it software? Right? Like, does it help to think about creating - Does this feel useful to any of you, the idea of creating a new category to put this in?

To name it? To say, “This is where this budget needs to go.” Because right now, if you're constrained in budget, in all of the areas that already exist in your business, and one of the things you say is, “Cannibalize your business line,” feels like we might need to cannibalize or even update the way we think of this as an investment category. Yeah, I completely agree.

It might take years for that to trickle into enterprises. I think the majority of companies that I meet with when they're talking again at that Xcom level, at the board level, they're looking largely at the employee salary category and also looking at the tool and tech procurement category. And that like includes cloud and compute spend, all of that kind of flows into office of the CIO. So they're still analyzing it there.

I think there's - we've had change management process reinvention things before. We had it with cloud. It's probably going to be treated in the same line items for that. What I do think changes is how you measure the efficacy of it.

And so maybe in that way you might want to pull out these millions, tens of millions, hundreds of millions of dollars depending on how large your company is. You might want to pull that out to be able to measure it differently. One, again, is employee productivity, but I would say you really want to look at other KPIs. Maybe it's like engineering velocity, maybe it's net new ideas that have been tested out, maybe it's how trustworthy your customers feel about you, or how fulfilled your employees feel.

I mean, there's so many other KPIs that I'd be looking at, but growth is the one that I think it should be measured on, not just productivity. Yeah. Tell us more about those “cannibalize your business” lines. We've talked a lot about this on this podcast.

Also the idea that you have to think, you have to approach this as business transformation as opposed to marginal improvements and all the stuff you're already doing. Yeah. And you have to do both, right. Like you're not going to be able to convince your board if it's just growth and you don't actually save money.

So I want to give, you know, productivity some love. But on the cannibalize your your business workflows like it is also about cannibalizing your tasks. There's this weird decoupling of human and the day to day that we had in 2019 that I think a lot of people are still grappling with, and so when I say cannibalize your business lines or cannibalize your workflows, or cannibalize your workflows, if you're a health care company or a fintech company, let's say, I was meeting with the real estate company, but it's a finance example.

They had one business type. That is their business type. One analyst, in his free time, decided to use generative AI to build out a brand new auditing product for the energy usage for their clients. They saved their clients 1 to 3%.

The reward, the business model, was that they took 50%. And this one person's idea has made millions in its first month. You might have to move some of your analysts away from whatever they're doing now to be able to reallocate toward net new business creation. So I think change doesn't feel good, and we are asking people to meaningfully shift how they spend 40 plus hours of their week.

It's going to take years for people to get their head around that, but cannibalization is like that first mindset shift to get around. the recent Microsoft Word Trend Index, the one that was just released, 65% of AI users, and I assume this applies to managers also, say they are afraid of falling behind if they don't move quickly, but 45% say it feels safer to focus on current goals rather than to redesign their work with AI. And, you know, people, employees and managers are caught between these two pressures.

What do you suggest? Give us the examples that just help with the storytelling to say, “I hear you, we're in a new world now, and here's what you could be doing.” Yeah. and I also want to say like leading by fear and saying, like, “If you don't figure this out in the next three months, you're out.

” never works. You'll have mutiny. There was a company that I heard about where they basically said that. And then, for the next two months, every single employee productivity plummeted because they weren't motivated to adhere to that vision.

That's a really important point that I want you to put a finer point on that from a management perspective, because that's what employees think that they are hearing, even if they're maybe not hearing it. There is so much product management that goes into this stuff, and I think, like - I was at a conference and the number of people that came up to me saying, “My boss has no idea what's going on. My management sucks. I feel like they're operating in a vacuum.

They don't understand my day to day. They think that AI can replace their entire marketing department, and it can't possibly do this part.” Whatever the thing is. A lot of it comes down to leadership faults.

The first thing that I would do is understand what is actually blocking people. Why are they fearful? Why are they leaning into that short term and not doing long term? Is it because they don't have enough time?

Is it because they're VP secretly in a meeting, you know, or over happy hour after one beer said, “AI sucks, I hate it.” Is it because they feel this reorg pressure and they don't want to jump the gun? Or they think that they'll use it wrong because you haven't given them a clear data privacy protocol? There's so many reasons that they might be blocked.

Step one is caring about your people, asking them questions, and not having this weird ivory tower separation because you are not going to bring people along for the ride. Number two is deciding to actually do it. I think a lot of leaders set this vision and then they go hands off and they're like, “Someone else can run this AI strategy,” which, by the way, happens to be you all in this room. And a lot of people, I think in 2023, were hiring chief AI officers, and I thought that was he worst idea ever.

Really? Yes! You are offloading the responsibility of your entire company strategy and reinvention to one person who, by the way, is probably a brand new hire and does not have the background of your company. So step two is having that that consistency of vision and really caring about and leaning in.

And then number three is like the moat of 2026 and beyond is speed of iteration. And enterprises are in a really difficult position because these new companies are hundred x-ing each individual employee. There's something called a lean AI leaderboard. Has anyone seen this?

So if you go to lean - I think it's literally lean AI leaderboard dot-com. You will see that there's a race of companies that are 50 people or less, and they're looking at the average revenue per head, average revenue per employee, and they are trying to hit 10 million-plus, 20 million-plus to be able to build a very small company that is valued at a billion-plus. That is what is happening, and so at the enterprise scale, you have to prioritize speed of iteration. That looks like figuring out these small experiments, having smaller teams, maybe fewer layers, lower bureaucracy.

It means allocating an actual budget that they know they can operate freely within. And then maybe a third, more funny thing is, like you should be able to have your people do things faster, like how many people dictate to AI in their in their day? A couple... Oh, okay.

Good. About half. I barely type. I am constantly, constantly dictating.

It is four times faster than writing. If all I did was dictate and the person next to me typed, I am in a much better position to be able to do anything to complete tasks. And so just giving your employees and teams the resources that are state of the art, that are frontier and giving them that space to play, again, as small as dictation software, as large as this entire frontier unit that's testing brand new models, that is what leaders have to do, and they have to start by caring about the people.

Yeah. Let's combine this idea of having agents work for you and with you with dictation as a hack. Right? Just the just the - because you gave me this great example backstage that I think really illustrates - Oh, yeah.

a totally different way of thinking about work day to day. Yes. If you have - like you guys are in Seattle, at least for now, like just go and reach out to the founders on Twitter and just ask to visit their headquarters. I just spoke with a founder that...

he - I was like, “You look great. Like, how are you... AI's crazy. Like, how are you so happy?

” He's like, “I walk 18,000 steps a day.” And I said, “How on earth are you doing that?” He has set up a voice back and forth with his lead AI agent. So it's worth talking about this like workforce setup just for one second.

-Yeah, no. Please, exactly. -So I have these... -That's what we want.

We want the detes. -Yeah. At these 34 agents he had a very similar set up. I have a chief of staff.

His name is Simon. Simon has, like, a memory documentation assistant named Toby. Simon has six direct reports. All the direct reports are named after the ‘Friends’ characters because I live in New York, and so I've got, like, Chandler running marketing, and Rachel running client work, and Phoebe was just the crazy, you know, Lucy Goose in the corner just been like, “Why don't we 20 exit?

” and those six have somewhere between... -Why don't we 20 exit? -Yeah. -Let's go.

Tagline. So and then they all have about like 3 to 7 sub agents and they can all spin up what I call like civilians which would be temporary agents, and maybe it's two agents or hundreds of agents. So let's just say we have those 34 agents, but I'm mostly talking to Simon, a chief of staff. This 18,000 step gentlemen had the exact same setup.

His chief of staff is Maya. So he has set up a live voice conversation with Maya because he doesn't even want to be looking at his screen. And so he just goes on steps and has full voice conversations, kicking off tasks with Maya. Maya delegates it out depending on, you know, whether it's an engineering task or a product task or a client task, being able to do that, that is the way that some people are working today.

I went into a headquarters of a very forward-looking Silicon Valley startup. Every single person had a $60 microphone at their desk, and they're dictating all day long in an open office setting right there, like whispering, you can't really hear it when you're a couple feet away. But that is what workforce transformation process transformation is looking like. They're in Teams, they’re in Slack, they’re doing all these things and they have agents sitting in Teams to be able to say, “Hey, you know, Deirdre, can you weigh in on this?

” They're having agents teach other agents new skills. This is why you have to believe that AI is not a tool. Because if you thought that it was a tool, would you think that you should set up a self-learning flywheel and have 60 agents kick off a thousand? Absolutely not.

We're having to treat it as this like operating system. You talked about like how do we describe this layer. -Yeah -I talked to executives that basically you have to spend some time building up an AI layer for your company. You have to make your whole company queryable in the AI age to be able to have this second brain and take action off it.

And I like I work with one exec who, at the end of every single day, she opens up her phone, she dictates to AI for somewhere between like five minutes, forty minutes, what key decisions were made, what is she happy about, what is she stressed about? You know, did John, like, really lean in in that meeting? Like, maybe you're recording meetings, but you're not capturing these like, human nuances that actually drive 80% of our decision making, right? If you have this unbelievable AI board meeting and everyone is talking about how great AI is and your general counsel is like this, staring at their phone the whole time, you need to capture that somewhere.

And so again, she's built up this daily context layer that she's been working on for months, and now she can go into AI and say, look at my notes for the last three months. Am I getting more stress, less stress, map out my energy, give me an entire web interface so that I can, you know, dynamically adjust how I spend my next three weeks so that I can have better outcomes based on the trailing three weeks. Building up this context, then be able to take action off of it, whether you're walking 18,000 steps or if you're like me, just like sitting and taking a 40 minute walk, that is a meaningful shift in how people are spending their time.

I'm laughing at the idea that we have now told you that you have to do all of this with the 34 agents and get 18,000 steps, like, this is a new world. -Yes, raising the bar. Okay, who is doing this? Like who is doing some version of this?

I mean, are you dictating your texts at least? Nobody has time for that. Yeah, okay. A couple people I know autocorrect is hard.

Okay. So we have we have room to grow for sure. I mean, I think this is - what you've just illustrated feels to me like the exact shift that you're talking about. Like, we just sort of caught up to using interactive LLMs and maybe developing agents for specific purposes.

Just like find your weirdos, like, find these people who are operating at the edge and I'm sure many of the weirdos are in this room. Maybe. You know, -We say that with love. -You're wearing your light blue button up today, but I know - And again they might be engineers.

They might be in H.R. They might be gamers in their free time. They might be Gen X, Gen Z like it is in every single corner.

But you have to figure out leadership ways to unearth these people. Maybe it's a hackathon. Maybe it's having people, you know, submit different ideas, maybe it's just looking at who's sharing things in Teams or town hall or whatever your mechanism is, but you have to have a mechanism to find the weirdos. I want to go back to money.

You have frequently cited over time, McKinsey's guidance for a 1 to 5 budget ratio. $5 on people for every $1 on AI tools. Does that still hold? First of all.

Yeah, I think it's a it's a helpful indicator of how much emphasis should be put on the people side, like in office of CIO world, which is leading a lot of the procurement of these tools, it is a very hard position for a head of IT to come in and weigh in on this, and I've found more often than not that the head of IT at least can't be the sole person that is running this. Like HR gets involved a lot, so that 1 to 5 ratio, the one is on literally paying for token. Like has anyone heard token maxing?

OK, lots of nods. -They’re like, “Yeah, we maxed. Help!” -Yeah.

So that one is the token maxing. How are we managing big models, small models, our architecture set up? Whatever. The five.

I think it's obvious that you'd spend money on upskilling. Like having that, you know, monthly update. It's obvious that you maybe, change your hiring processes and you need to retrain your people managers. The less obvious dollar that is being spent is on incentives, and money incentives are very strong.

And so maybe if you are running this hackathon to get a new multi-agent or agent system ideas out there, maybe the reward is money, right? That is still very much a people reward, but it is not just going on upscaling. You like sneak in the upskilling into the hackathon. And in doing so, you have upscaled your whole org, but you're allocating this money to make sure that it rolls out with everyone.

this is pre, you know, big generative AI, but, I was working with a company and they had a lot of different retail locations. They had figured out this unbelievable way of managing inventory. It was going to be perfect. It is the greatest prediction system ever.

And they roll it out. No one used it, right? Not a single person used it, and they had spent millions of dollars creating this. Zero adoption.

And only after seeing zero adoption did they then go to the retail location owners to say, “What would you like in this product? Why aren't you using it? What would make you earn trust with it?” Right?

I worked on another product that was talking about computer vision inside of manufacturing facilities, and no one thought that you couldn't hear someone in a manufacturing facility, and so every single thing was going to be voice-enabled. You can't do that when the machines are crazy loud. So just bringing those people in early because they're going to catch things that you don't, that is still spending those dollars in that 5 to 1 ratio. In your experience, when a Fortune 500 actually commits to this, not a bolt on, but truly a rewire, what is the first thing that has to change?

Assuming that the CEO has like used AI, which by the way, there are still CEOs who have never opened generative AI systems. But let's just assume in 2026, let's assume that it's a company where the CEO is bought in. I think that the number one thing to pay attention to in the AI age is the pace of change. And I talk to leaders and I think that they feel the nausea, like they feel the acid reflux, but they don't quite see the exponential.

And so the number one thing that I would do is just like, make sure that you understand the exponential so that you can at least somewhat better predict the next 1 to 6 months. So as an example, on coding benchmarks, the recent frontier models that have come out on our coding benchmarks, they are hitting 94. Okay, a year ago, we were hitting 72 with these frontier models, and a year before that, we were at... anyone?

Audience? Seventeen! If you're making a multi-year procurement strategy, if you signed a three year, you know, tech deal, that is where we are two years later. And so I think, like step one is you have to wrap your head around the exponential pace of change, because then instead of building these more manual systems, instead of relying on the way that you get work done today in the exact same team structure, you have today with the exact same budget allocation you have today, you're naturally whether it's fear in like, induced or joy induced, you're going to feel that you have to change.

So making sure that CEOs or the ex comma in general sees that is very, very important, and is almost impossible to predict, you know, even one year out. But better planning the few months and working with an advisor, whether it's me or someone else or a provider like Microsoft, you need to know what's coming around the corner because it, in an enterprise space, takes months to years to change process. Well, then say more tactically about the the frontier team idea that you've talked about forming a frontier unit.

These cross-functional teams like what does that look like? Who should they report to? What should you know? What should leaders be borrowing from those teams that aren't you know, that, again, aren't necessarily pilots, but at least they're in charge of this so that they can seed change as it happens?

It has to be cross discipline. I do not want to see a frontier unit of all engineers. I want to see research. I want to see the weirdos in marketing, in sales, in legal, in finance.

I want them to be represented because you never want these ideas or prototypes or roll outs to die when you go to a department, you want to be able to say, “Hey, Susie's already checked this. Devin's already checked this. They said that they're in. Here are the three reasons why.

” You want to have that alignment loop happening fast. So for sure across discipline. The second is it's probably not that much bigger than 120. Like, even if you are at a very large company, it needs to be a separated out group.

They need to have almost no layers of management. It's like maybe one layer in between. And it is also small teams. Like every company I work with, the best ideas started in a team of 2 to 8.

And then that frontier unit going back to token maxing and the thing we said in the very beginning, they probably have much higher budgets, much higher budgets again per head. And so right now, again, you might be in a position where even a small team within the frontier unit is getting thousands of dollars per day to figure out again, ten iterations of what is working best so that you take the one winner and then you make the efficiencies come out of that. As far as who runs it, it is different for every company I work with.

Some it kind of comes out of like the R&D innovation zone. The second might be that you just have one C-suite executive who is so brought in, and it might be the CFO. I worked with some brilliant CFOs, CMOs who are non-technical, who are leading these initiatives. But it is always a, passionate weirdo who is using this in their free time, and probably suffering from sleep .

-Right. -Sleep loss. -Yup Okay, and then finally, our last two minutes here, everyone is going to be heading back to their organization. This is the last day of the summit.

In no way would I ever ask you to project three years out. I think we have all learned. That's not happening. -The labs don't predict even more than six months out yet.

-Let's talk about 30 days. -Okay. -In the next 30 days, what can the people who leave this summit do to move from AI curious to AI first and, two minutes. What should they stop doing?

So we talked about building up that AI layer to make your whole company queryable. I would do every single thing in my power to crack the context puzzle. Are you connecting it into tools, SharePoint, Outlook? What are you connecting it into?

Are you having your meetings recorded and transcribed even if you're in a regulatory environment? Are you having your executives dictate these things? Are you allowing for your employees to build up context docs? Right?

I have my AI interview me to build up context doc. So context, context, context. We are in the era of context engineering. Second, you have to have AI work proactively.

You should have AI prompting you for things. You have AI coming to you, whether it's scheduled or trigger based or time based or something. It has to be coming to you. If you are the only thing that is kicking off AI workflows, you are three x already because you're only working eight hours and the 24 hours in a day.

The thing that I would stop doing, I would stop typing. I know that sounds insane and I say this because then I think you're going to reduce it by 10%. Like I have to set the bar high to then let's shoot for the stars, settle on the moon kind of thing. I want people to stop typing.

I want people to move into crazy multimodal, gremlin-level communication, giving images, giving dictation, being able to set it up to actual software systems, building that new interfaces so that decisions, even in meetings, become more dynamic. I would be building up that context layer, getting way more proactive so that AI is functioning as an operating system. Amazing. Thank you so much to Allie K Miller, CEO of Open Machine and one of the most influential voices in AI today.

I think you can see why I spend more type less. Thank you all for joining us live at the Copilot Summit. For more conversations from the frontier of AI at work, please follow the show and listen to past episodes wherever you get your podcasts. I’m Molly Wood, thanks for listening.