A Conversation about the Human - AI Teaming Landscape: Designing the Hybrid Workforce

Speaker A: Welcome to today's Deep Dive. So, um, think about your job for a second. Think about the tools you use every single day.

Speaker B: Right.

Speaker A: Because for the better part of the last decade, whenever executives or managers looked at their workforce and looked at new technology, they were asking, you know, the exact same question.

Speaker B: Yeah. They'd look at a process and ask, what can we automate?

Speaker A: Exactly. What can we automate? It was always the substitution game. The goal was just to swap the human out, put the machine in and cut costs.

Speaker B: Cut costs and boost efficiency. Yeah.

Speaker A: Right. But then in late 2020, with the whole explosion of generative AI, that entire playbook was just completely thrown out the window.

Speaker B: It really was.

Speaker A: So the mission of our deep dive today, which is guided by this really brilliant research article by Dr. Jonathan H. Westover. Um, it's called the Human AI Teaming, Designing the Hybrid Workforce.

Speaker B: A great taper.

Speaker A: It really is. And the mission today is to show you why the fundamental unit of work is no longer the individual task. The new unit of work is the team. Specifically, humans and AI systems working together under, like, a shared accountability.

Speaker B: What's fascinating here is that Dr. Westover makes it incredibly clear that we are no longer dealing with a technology problem.

Speaker A: Right.

Speaker B: Because the foundation models out there, they're converging. Right. They are widely available and just staggeringly powerful.

Speaker A: Yeah. Everyone has access to them now.

Speaker B: Uh, exactly. So what we have now is an organizational design problem, hybrid performance. I mean, getting a human and an AI to actually outperform either one, working completely alone, it's absolutely not automatic.

Speaker A: No, definitely not.

Speaker B: So the ultimate goal of this research is to figure out exactly how we design these hybrid workflows so the human and the machine complement each other rather than, you know, mislead or de skill each other.

Speaker A: Yeah. And to even begin to understand this human AI teaming, we have to completely strip away our old mental models. Like, we just have to stop treating AI like it's just another piece of software.

Speaker B: Absolutely.

Speaker A: It is not an upgraded version of Microsoft Excel, and if you treat it like one, you are walking right into a trap.

Speaker B: Yeah. The researcher, Sieber and colleagues, they make a really vital distinction on this front. They say AI is a teammate, not a tool.

Speaker A: A teammate. Okay.

Speaker B: Yeah. Right. Because a traditional tool like a spreadsheet, it just executes a formula. It does exactly what you tell it to do. Nothing more, nothing less.

Speaker A: Right. If you make a mistake, the spreadsheet just reflects your mistake state.

Speaker B: Exactly. But an AI teammate, on the other hand, proposes entirely new options. It generates novel artifacts, and crucially, it has the capacity to be confidently wrong in ways that looks syntactically and logically

Speaker A: perfect, which is terrifying.

Speaker B: It really is. And that fundamentally shifts the cognitive and social demands placed on the human being working alongside it.

Speaker A: Okay, let's unpack this, because I see organizations making this mistake. Constantly treating an AI like a calculator is deeply dangerous. Because a calculator is never confidently wrong about math.

Speaker B: No, never right.

Speaker A: It either gives you the exact answer or gives you a syntax error. AI is much more like an incredibly fast, highly educated intern who never sleeps.

Speaker B: Great analogy.

Speaker A: But this intern occasionally hallucinates facts with total unwavering confidence.

Speaker B: Yeah, that analogy captures the dynamic perfectly. And it brings us to this landmark study by Delacqua and some researchers from Harvard and the Boston Consulting Group. They studied highly skilled consultants using GPT4, and they mapped what they call the jagged technological frontier.

Speaker A: The jagged frontier. That visual is so sticky. Explain how that actually looks in practice.

Speaker B: Well, imagine a boundary line of competence. When a task falls inside the AI zone of competence, meaning it's a task the model handles well, like synthesizing information or maybe drafting standard reports. The results are staggering.

Speaker A: Staggering how?

Speaker B: The researchers found that consultants using AI completed 12% more tasks. They did them 25% faster. And the quality of their work was evaluated as 40% higher than the control group who weren't using AI.

Speaker A: Oh, wow. 40% higher quality.

Speaker B: Yeah, it's massive.

Speaker A: I mean, any manager listening to those numbers is going to instantly mandate AI across their entire department. But you called it a jagged frontier, which implies there are, like, sharp edges,

Speaker B: there are absolute cliffs. Because when the researchers gave those exact same consultants a task designed to sit just outside that frontier of competence, like a task requiring deep nuance or logic that the AI struggles with, what happened? The consultants using the AI were 19 percentage points more likely to produce completely incorrect answers than the control group.

Speaker A: 19% worse. That is a huge drop.

Speaker B: Yeah, because the AI sounded so authoritative that it confidently led these highly educated professionals right off a cliff.

Speaker A: So for you listening to this, I mean, the immediate actionable takeaway is that tracking the average effect of an AI rollout in your company is, ah, a totally useless metric. Completely useless because an average masks the underlying reality. Right. You have to track the distribution. You have to map out exactly where your specific AI acts like a brilliant assistant, and exactly where it acts like a confident saboteur.

Speaker B: Yes. If you are leading a team, your operating model must be able to detect that difference in real time.

Speaker A: Right.

Speaker B: Because since AI is a teammate with this highly unpredictable jagged Frontier. We naturally have to ask the next logical question.

Speaker A: Yeah.

Speaker B: Who on the human team actually captures the most benefit from working with it?

Speaker A: And the data on this is completely counterintuitive. It's known as the productivity asymmetry.

Speaker B: Yeah. This is fascinating.

Speaker A: There is a massive study by Brynjolfsson and his colleagues looking at over 5,000 thousand customer support agents. They integrated a generative AI assistant into the workflow and the overall average productivity gain was 14%.

Speaker B: Which sounds good on the surface, right?

Speaker A: It sounds great. But when you break that average apart, the novice workers, the brand new or low skilled agents, they saw a, um, massive 34% gain in productivity.

Speaker B: 34%.

Speaker A: Yeah. But the top performers, the seasoned experts who'd been there for years, they saw almost no change at all.

Speaker B: None. And we see that identical pattern replicated in a study by Noi and Zhang focusing on professional writing tasks. Oh, yeah. The AI dramatically speeds up completion times and raises the overall quality floor, but it does so specifically by narrowing the gap between the lower performers and the experts.

Speaker A: So the lower performers are just catching up faster.

Speaker B: Exactly. They rapidly catch up.

Speaker A: Yeah.

Speaker B: What is happening under the hood is that the AI is essentially transferring the tacit knowledge of the experts directly to the novices.

Speaker A: How does it do that?

Speaker B: Well, the model ingests what a masterful, empathetic customer response looks like from a veteran and then feeds it as a real time suggestion to the person who just got hired yesterday.

Speaker A: Here's where it gets really interesting though, because as I read that, I couldn't help but push back on the long term implications of this.

Speaker B: I know exactly where you're going with this.

Speaker A: Right. Because if novices are utilizing AI as a shortcut to bypass all the hard, messy parts of learning a job, are we risking massive, widespread deskilling across the workforce?

Speaker B: Yes.

Speaker A: How does a novice ever develop expert judgment if they never actually have to struggle through a complex problem on their own?

Speaker B: This raises an important question, and it is arguably one of the deepest concerns in the management literature right now.

Speaker A: It's scary to think about.

Speaker B: It is. Researchers Raiche and Korkowsky define this as the Automation Augmentation Paradox.

Speaker A: The automation augmentation paradox.

Speaker B: Okay, so if an organization leans too heavily into just automating the early formative parts of a profession, they slowly erode the very human capabilities they will eventually need to govern the AI.

Speaker A: Oh, wow.

Speaker B: Think about a law firm, right? If the junior lawyers never grind through routine document review, they never develop the granular pattern recognition required to eventually become senior partners.

Speaker A: That makes total Sense.

Speaker B: And you need those senior partners to oversee the AI when it's drafting complex legal strategy down the line.

Speaker A: So the expertise pipeline just dries up completely. You get a generation of workers who know how to accept prompts, but don't know the underlying principles of their own profession. And that risk of deskilling leads us directly into the biggest psychological trap of hybrid work, which is how do we actually calibrate our trust in this non human teammate?

Speaker B: Trust calibration is the crucial phrase.

Speaker A: Yeah.

Speaker B: Gligson and Woolley synthesized empirical research on human computer interaction and found that human trust is almost never accurately matched to the actual reliability of the system.

Speaker A: We're terrible at it.

Speaker B: We really are. We rarely find the middle ground. Instead, we swing wildly between two extreme behavioral reactions.

Speaker A: And the first extreme being what researchers Log and colleagues call algorithm appreciation.

Speaker B: Yes. And it is incredibly common with generative models.

Speaker A: Right, because they sound so good.

Speaker B: Exactly. This occurs when users vastly overtrust the AI because the output is fluent, syntactically perfect, and sounds authoritative. Humans conflate eloquence with accuracy.

Speaker A: We just assume if it sounds smart, it is smart.

Speaker B: Right. We engage in cognitive offloading. We subconsciously decide the machine has it handled and we just stop checking the work.

Speaker A: And then you have the exact opposite extreme, which is Algorithm aversion, studied extensively by Dietforst and colleagues.

Speaker B: Yes. The aversion side.

Speaker A: This is where a user sees the AI make a single mistake, just one hallucination or error, and they completely abandon the tool forever.

Speaker B: They just drop it?

Speaker A: Yeah. They decide the machine is useless and determine they can only rely on themselves.

Speaker B: Both of these miscalibrations are incredibly expensive and dangerous for an organization.

Speaker A: I can imagine.

Speaker B: Yeah. Over trust leads to catastrophic errors passing through to clients. And under trust means you are leaving double digit productivity gains completely on the table.

Speaker A: So think about this in terms of your everyday life. Like using a GPS navigation app.

Speaker B: Oh, that's a perfect example.

Speaker A: Right. Algorithm appreciation is driving your car straight into a lake because the app said turn left in a confident voice and you completely suspended your own physical senses. And then algorithm aversion is sitting in an hour of bumper to bumper gridlock because the app took you on a weird detour one like three years ago. So you refuse to ever open it again and rely entirely on your own flawed sense of direction.

Speaker B: We've all done that.

Speaker A: We definitely have. So if human nature naturally defaults to either the lake or the gridlock, how do real world organizations actually fix this behavior?

Speaker B: Fixing these extremes requires highly deliberate organizational design. Dr. Westover's research highlights several case studies of companies building out the necessary surrounding systems to manage this.

Speaker A: Uh, like who?

Speaker B: Let's look at communication and transparency first. When GitHub rolled out its co pilot coding assistant to its own internal engineers, they didn't just quietly install it and tell everyone to be more productive.

Speaker A: Right.

Speaker B: They openly published controlled studies on developer productivity. But more importantly, they explicitly published the map of where the tool helps and where it hurts.

Speaker A: Wow. So they were totally honest about its flaws.

Speaker B: Yes. They framed the AI strictly as an assistant whose suggestions the developers were expected to critically evaluate.

Speaker A: Which completely changes the psychological dynamic. Because if leadership doesn't communicate clearly about the tool's flaws, employees start hiding their AI use. Or worse, hiding the errors because they're

Speaker B: terrified of being replaced.

Speaker A: Exactly. But simply telling an employee to evaluate the AI assumes they actually have the skill to spot a plausible sounding error. Are companies actively teaching that capability?

Speaker B: The smart ones are. Look at Moderna's approach to capability and AI literacy.

Speaker A: What did they do when they rolled

Speaker B: out ChatGPT across their enterprise? They didn't just offer some generic, optional intro to AI webinar on a Friday

Speaker A: afternoon like so many companies do.

Speaker B: Right. They treated prompt engineering and output verification as core professional skills, and they embedded that training directly into daily workflows.

Speaker A: Oh, uh, that's smart.

Speaker B: Wasn't taught by it. It was taught within the context of specific scientific and business tasks. Yeah, you have to actively teach your workforce how to interrogate the model.

Speaker A: Okay, so the employees are trained to spot errors, but if the AI is doing the bulk of the initial drafting or coding, how does that change the actual structure of the human's daily job?

Speaker B: This is where governance and procedural justice come in. JP Morgan Chase built a system called Koyan to analyze commercial loan agreements. They didn't just automate the reading of the contracts and leave the lawyers to figure out what to do with their free time.

Speaker A: Right, because that never works out well.

Speaker B: Exactly. They set up a very narrow scope with tight feedback loops. And they entirely redesigned the roles of the human lawyers. The job shifted from reading hundreds of pages of boilerplate to focusing almost purely on exception handling and auditing the anomalies that the AI flagged.

Speaker A: So what does this all mean? Because when you describe reshaping a job around the AI, we have to look at the tension in the Klarna example.

Speaker B: Yes, the Klarna case is a big one.

Speaker A: Klarna, the financial services company, deployed an AI assistant that handled two thirds of their customer service chats in its first month. And the satisfaction scores matched human agents.

Speaker B: Incredible numbers.

Speaker A: But that rollout coincided with a broader workforce reduction. Let's stop there. If I am an employee listening to this, Klarna just proved my biggest fear.

Speaker B: Right? The fear of replacement.

Speaker A: They didn't build a hybrid team, they built a replacement model. If an organization just slaps AI onto a workflow to cut headcount, aren't they just building a surveillance state where the remaining humans feel intensely monitored and redundant?

Speaker B: If we connect this to the bigger picture, you are hitting on the fundamental rule of this transition. Tools change tasks, but only organizations change work.

Speaker A: That's a great way to put it.

Speaker B: A job is just a bundle of tasks. If you use AI to automate half the task and just fire people without rebundling the remaining work, you get exactly what you described.

Speaker A: High surveillance nightmare.

Speaker B: Right. An environment where employees feel disposable and you retain all the old organizational friction. You have to deliberately design for what researchers Dougherty and Wilson call the missing middle.

Speaker A: I want to make sure we really understand that. What does the missing middle actually look like in practice?

Speaker B: The missing middle represents the entirely new categories of roles where humans train, sustain and and explain the AI, while the AI amplifies and embodies human work.

Speaker A: Give me an example.

Speaker B: Well, in a customer service context, it means the human is no longer just typing answers to password reset questions.

Speaker A: Because the AI does that now.

Speaker B: Exactly. Their job is elevated to a conversation designer. They are curating the training data, handling deeply emotional customer escalations that require genuine empathy, and auditing the model for bias.

Speaker A: I see.

Speaker B: If you don't redesign the work to create and value that middle ground, the human AI collaboration ultimately fails because you've hollowed out your human capital.

Speaker A: But how do you enforce that middle ground? I mean, how do you guarantee that a busy employee actually exercises judgment rather than just blindly clicking approve on the AI's output?

Speaker B: To save time, you have to build in trust controls. Specifically human in the loop systems. Mayo Clinic's deployment of clinical AI is the gold standard for this.

Speaker A: What's their approach?

Speaker B: They use AI across imaging and clinical workflows, but they purposely keep human clinicians visibly and structurally in the loop for all high stakes outcomes.

Speaker A: Okay, so the doctor's always involved.

Speaker B: Always. They frame the AI strictly as decision support. The user interface doesn't just have an approve all button.

Speaker A: Oh, that's key.

Speaker B: Very key. They build in forced friction checkpoints. The doctor has to physically engage with the AI's recommendation and validate the specific anomaly to prevent that algorithmic complacency.

Speaker A: This tension of redundancy, of feeling like the machine is taking over the core artifacts of your profession. It brings us to the final and I think the deepest impact of Dr. Westover's research.

Speaker B: A psychological impact.

Speaker A: Exactly. What happens to the human worker psyche and their fundamental identity when the AI starts doing the heavy lifting?

Speaker B: It creates enormous identity strain, particularly for knowledge workers whose self worth is tied to their output. Yeah, we have to talk about recalibrating the psychological contract.

Speaker A: The psychological contract, yeah.

Speaker B: Uh, for decades, the implicit deal between employer and employee was straightforward. The company provides the infrastructure, and the worker provides the judgment and the creative output.

Speaker A: But generative AI blurs that line completely.

Speaker B: It shatters it.

Speaker A: Honestly, think about your own job right now. You know, if AI starts taking over the actual artifacts of your work, if it drafts the legal documents, writes the software code, designs the marketing graphics, crunches the quarterly spreadsheets, what are you actually there for?

Speaker B: It's a crisis of meaning, right?

Speaker A: What is your value?

Speaker B: Workers are reasonably asking profound questions like, if the corporate model learns my unique style from the templates I created, who owns that intellectual property?

Speaker A: Such a good question.

Speaker B: And if this AI makes me twice as productive, do I get to go home at 2pm Do I get a massive raise?

Speaker A: Probably not, right?

Speaker B: Or do you just fire half my department and expect me to do the work of two people for the exact same pay?

Speaker A: And without explicitly recalibrating that unwritten psychological contract, employees will just write the narrative in their own heads. And let's be honest, in a corporate vacuum, employees almost always write a pessimistic narrative.

Speaker B: Always. Which destroys psychological safety. Harvard's Amy Edmondson has proven time and again that teams that feel psychologically safe, teams that feel secure enough to ask stupid questions, report their own errors, and experiment without fear of surveillance or punishment. Yeah, they vastly outperform teams driven by fear.

Speaker A: Yeah, fear is a terrible motivator.

Speaker B: Long term, it is. And generative AI should ideally lower the cost of asking a stupid question, right? You can ask the bot instead of your boss.

Speaker A: Oh, that's true.

Speaker B: But if the AI is deployed as a surveillance tool that scores every interaction to measure efficiency, it suppresses that learning entirely.

Speaker A: So how do leaders actually resolve that identity crisis for their team? I mean, when the tangible artifacts of the work are being generated by a machine, what is the core purpose left for the human?

Speaker B: The answer lies in pivoting organizational focus toward what AI intrinsically cannot satisfy. Organizations need to redefine human work toward judgment under intense ambiguity, toward genuine care, relationship building, and empathy for Stakeholders.

Speaker A: Things machines can't do.

Speaker B: Exactly. Toward the integration of competing, highly nuanced human interests. Above all, toward accountability for the final outcomes.

Speaker A: The accountability piece is huge.

Speaker B: It's everything. An AI cannot be held legally, ethically or morally accountable for a business decision or a medical diagnosis. A m human being must be.

Speaker A: And building that specific capability requires continuous learning systems. You can't just treat an AI rollout as a finite IT project, ship it and forget it.

Speaker B: No, because these models drift over time.

Speaker A: Exactly. Regulations change, competitor practices evolve. You need quarterly reviews, blameless postmortems on AI incidents, and distributed AI literacy, where the frontline workers, the people actually doing the job, are empowered to make decisions about how the tool is governed.

Speaker B: The compounding advantage in the future of work goes exclusively to organizations that learn faster than their tools change.

Speaker A: Which brings us to the ultimate synthesis of Dr. Westover's paper. The massive AI engines under the hood, the foundation models. They're converging. They are becoming commoditized. Within a few years, every company on earth will have access to the exact same, incredibly smart, occasionally hallucinating intern. The technology itself will not be your competitive differentiator.

Speaker B: No, the ultimate advantage goes entirely to the organization that builds the best surrounding system.

Speaker A: Right?

Speaker B: The governance, the continuous learning protocols, the psychological safety, and the highly deliberate rebundling of work design.

Speaker A: So to you, the listener, evaluate your own workflow tomorrow morning. Ask yourself, are you actually managing the jagged distribution of your AI's helpfulness? Are you tracking exactly where it is brilliant and where it is leading you off a cliff?

Speaker B: It's a crucial self audit.

Speaker A: Or are you just blindly hoping for a blanket average improvement? Are you aiming for carefully calibrated trust with forced friction? Or are you just celebrating maximum adoption because everyone in your department happens to be logging in?

Speaker B: It is a critical distinction to make. Adoption metrics just tell you if the tool is being used. Calibration metrics tell you if the tool is being used.

Speaker A: Well, that's a great point.

Speaker B: Human AI teaming is at its root a leadership and organizational problem long before it is a technology problem.

Speaker A: And that leaves us with a final massive question to mull over. If this missing middle of human AI collaboration truly works. Like if we navigate the jagged frontier, solve the trust calibration and unlock these double digit productivity gains across the board, it begs a profound question about the future of society.

Speaker B: It really does.

Speaker A: Remember that 10 year obsession with automation to save time and money? Well, if we actually achieve this incredible efficiency, will this finally be the Death of the 40 hour workweek giving us our precious time back.

Speaker B: Wouldn't that be nice?

Speaker A: Or will human nature just do what it always does and invent entirely new, exhausting categories of work to fill the exact amount of time? Our AI Teammates just saved us something to think about the next time your AI Finishes a task for you. Thanks for joining us on the deep dive.