Episode 104: [Value Boost] The Four Zones of AI Productivity for Data Scientists

Dr Genevieve Hayes: Hello and welcome back to Value-Driven Data Science, where data professionals become strategic experts. I'm Dr. Genevieve Hayes, and I'm here again with Brent Dykes, author of Effective Data Storytelling. Last week, Brent and I discussed how data professionals can move beyond generating interesting findings to creating data insights that drive actual business change.

Today in this value Boost episode, we're exploring how AI is changing the process of generating insights altogether. Welcome back, Brent. Brent Dykes: Hey, it's great to be back. Dr Genevieve Hayes: In a recent article in Forbes, you introduced the four zones of AI productivity, a framework that maps out what happens when humans and AI collaborate on a project from the initial prompt all the way through to a finished high quality output.

Your central argument is that AI dramatically accelerates the early stages, but that human expertise becomes increasingly critical as the work progresses. What I'd like to do today is take that framework and apply it specifically to the insight generation process, because I think it has some really important implications for how data professionals should be thinking about AI as a tool. So to begin with, can you give our listeners a quick overview of what the four zones of AI productivity are?

Brent Dykes: The first zone is really I call it the launch zone. So we go from zero to 60%. When we think about the magic of ai it's able to, from a low human effort, we're able to generate something that's maybe 60% there in a matter of minutes. Or seconds in some cases, minimal prompting minimal effort from our side and whether we're a novice, intermediate or expert, we can all get there very quickly.

But then as we start to get into what I call the rework zone, where it's 60% to 80%, the human. Effort increases, maybe we have to actually apply some of our own skills to get that done. Obviously from a novice perspective, somebody who doesn't know a lot about the topic is gonna have to put in a lot more work to get something that actually moves up to from 60%. To 80%.

The next level is what I call the forge. And that's 80 to 90%. And that might be where a novice cannot operate they just don't have the knowledge of what good looks like to the point where they could actually achieve it. And maybe an intermediate person would still require a high level of effort and even an expert, it's gonna take some demands to get there.

And then the last level, which I call the summit, this is like peak performance. This is going from 90 to a hundred percent. A novice can't operate here, even in intermediate, can't operate here. It's only the expert level that can actually operate at the summit level, and so those are the different levels.

And in some cases, maybe there's a task or a project that doesn't require. A hundred percent. Many things don't require a hundred percent quality. Maybe 80% or 70% is passable.

So there's different situations, different levels and then obviously our own expertise is gonna play a role in whether we can even get to the summit or not. And I think the key thing is that everybody can produce something with ai, but. How far you can actually take it will be limited by your own expertise. And one of the core messages of the article that I wrote is that if you have the expertise in an area, you can achieve so much more with AI than somebody who's just a novice.

Dr Genevieve Hayes: So taking this framework and applying it to the insight generation process, you described that first launch zone stage as the stage where AI can do most of the heavy lifting and get you to 60% very quickly. When it comes to insight generation, what does that 60% look like? Brent Dykes: I think it's looking at where can. The strengths of AI help us in our analysis, I was thinking about it yesterday that, I can create a lot of visualizations that are very powerful and I know how to, tailor them and get them to a level where they're gonna do really well in Adidas story.

But in an analysis stage, I don't really need perfection, i'm the customer. And if the axes aren't perfect and the labeling isn't perfect and maybe the visual colors aren't the best. It doesn't really matter. I'm just fishing for insights.

I'm trying to get there quick. And so the rapid ability for me to generate multiple visualizations all at the same time, maybe with different variables, different metrics on a very quick basis. I'm perfectly okay with rough versions of those visualizations and. Once I've found something, then I can worry about how good those visualizations and I probably abandon AI at that point.

'cause fiddling with trying to get it to do exactly what I want it to do may be very hard to get it to a shippable polished. Visualization that I can put in my presentation. But up until that point where it's more about speed, it's about flexibility. And some of these visualizations are very complicated to set up.

And if I can use ai, even though imperfect, it can set it up and I can test something or I can start analyzing something again, I don't need perfection on the final polish yet. That may be a great way to go. And so I see ai, I think if we use it the right way and in the right places play to its strengths it could be a huge time saver for us. Then also recognize where its weaknesses are on that final polish stage sometimes that's where it's gonna struggle, and that's fine.

I can step in at that point, take over and get the polishing, get the final look and feel looking good. But that heavy cycling time of trying to find an insight in the data, partnering or collaborating with AI or ai augmenting what I can do seems like a great approach. Dr Genevieve Hayes: Is AI actually capable of identifying an actionable insight, or is it just capable of. Producing an visualization from which you could spot the insight.

Brent Dykes: I never wanna say never, but it would need a lot of training data to really know. What, context around what's important, how to analyze everything, having the full context of what's going on in the background. So it would take a lot of preparation to really train it up, to give what I would consider an actionable insight. I'm more likely to say it's gonna help me to find an actual insight, not generate an actionable insight on its own.

Dr Genevieve Hayes: So it's sounding like that launch zone is the equivalent of how a graduate data analyst would handle a problem. They'll produce a. Couple of hundred visualizations, and then point out the ones where, hey, look, there's this trend here, or this bar's a lot bigger than the others. And then at that rework and forge zones, you are bringing in your own human expertise and that domain knowledge in order to say these are the things that are actually.

Of interest to my stakeholders because they connect back to the business problems, et cetera. Brent Dykes: Somebody made a comment on one of my LinkedIn posts and this was him summarizing what I was saying, and I thought it was very insightful what he said. He said, treat AI less like a consumer, meaning us being the consumer of its output and handle it more like a manager. So you're overseeing, you are not just accepting what it's giving you.

And yeah you're working with a junior analyst essentially and I did a post last week that was interesting. I find when you're working with a human high performer, occasionally you'll be surprised by something really cool that they come up with. And then if you're working with a low performer. You prepare for oh, this person's gonna screw up two or three times.

So you're on guard and you're queing their work and oh, Jim always does this. I'm gonna double check that he did this. Oh yeah, sure enough, he didn't calculate it the right way. And the interesting thing about AI is that we get both.

Sides, sometimes we're just like blown away by what AI can do, and then other times you're just like, W wait a second. Why did you revert to this other data? We already clarified we don't want to use that data, and now you've done an entire analysis. Or why would you calculate it that way?

That makes no sense. Like completely boneheaded. Why would you do that? So our trust system is broken with ai 'cause we treat it like a high performer when it's completely brilliant and then baffling on the next moment.

We can't treat it like a human being. It performs un humanly. And we're just brand new at interacting with artificial intelligence. We as humans haven't been doing this for years and years.

And the only kind of touchstone that we have is how we interact with other people. And it feels human-like when we interact with, the comments and how it interacts with us. But we have to realize that it behaves inhumanly and we have to have our guard up and we have to treat it in a new way that's different than our coworkers and people that we work with on a daily basis. Dr Genevieve Hayes: So your article concludes by saying that better AI tools won't close the capability gap between experts and novices, but will actually amplify them.

What does that mean for data professionals who want to position themselves as strategic advisors? Brent Dykes: I think a lot of businesses and leaders are looking at AI as a way to replace human beings and. In some cases, the way I'm viewing it is if you just look to replace bodies with ai, there's gonna be a limit to what AI on its own can achieve and the expertise that you have. In certain employees can be immensely valuable and can really amplify what we're doing with ai.

And without that amplification, without that base expertise it's gonna be hard for you to really achieve. As much as you could by just doing AI alone. A lot of companies made mistakes where they said, oh, we can move all of our support functions and coding to ai. And now, in both cases we're seeing examples where companies are quickly trying to hire back people that they let go 'cause.

It's not as simple as that. I think we're more intertwined than we think maybe we need to be. And we are. You can't completely rip one or the other out and replace it with, humans.

You give up something if you just use humans, you give up something if you just use ai and it's really about collaboration between both groups. Dr Genevieve Hayes: It sounds like if you were to replace all the humans with ai, it'd be the equivalent of replacing all your senior data scientists or programmers with graduate programmers. The graduates could do something, but they're not gonna be able to perform at that same level as the more senior people. Brent Dykes: Yep.

I. Dr Genevieve Hayes: So given all of this, how should data professionals actually be using AI in their insight generation process? Brent Dykes: The basic level would be how can we take away those laborious tasks that are time consuming? I think of like data cleansing.

If there's certain things that we need to do from a data cleansing or preparation perspective, I think a lot of those obviously with oversight can be automated and streamlined using ai. Let's play to the strengths of ai. What can it do? It can replicate multiple tasks and do them simultaneously at high scale and repeatable tasks can be done quickly and efficiently and effectively.

And then we can then devote our time to those tasks where we know that, whether it's gathering context, whether it's, connecting dots between different things where we have an understanding of the market, landscape. We understand personalities in our organization whether they'll go for something or not. There's gonna be certain things where AI's gonna struggle 'cause it doesn't have the full context, doesn't have the full understanding of the audience and the goals behind and the politics and all of those wonderful things that.

Make our lives interesting that sometimes we hate. Let's play to the strengths of the ai. Let's bring it in and leverage it as much as we can, and then obviously know where there's limits to what it can do and how it can help us. And then hopefully free up our time so we can invest more time in those areas so that we don't have to do the laborious manual stuff.

That can consume hours that could be better spent on other activities. Dr Genevieve Hayes: That's it for today's conversation with Brent. If you haven't already listened to our previous episode where Brent and I discussed how data professionals can move from generating interesting findings to creating actionable insights, you'll find it at value driven data science.com or on your favorite podcast platform.

Thanks for joining me again, Brent. Brent Dykes: Great to be here and have a great rest of your week. Dr Genevieve Hayes: And for those in the audience, thanks for listening. I'm Dr.

Genevieve Hayes, and this has been Value-Driven Data Science.