Series 6 Episode 9 - Artificial Intelligence makes CRO better if you know your sh**!

Speaker A: You're listening to the Big Lift, the podcast of WebTrends Optimize, the CRO solution that enables marketers and developers to maximize the ROI on their digital properties. WebTrends Optimize is a powerful feature rich and easy to use solution, all delivered within a fixed price contract with no additional cost for increased functionality ever. During these podcasts, we meet some of the key influencers within the marketing and conversion world to understand their roles and examine their challenges. Today, I'm in conversation with Johan Van Tonda, who last appeared on the Biglift podcast over 18 months ago, where we discussed how CRO needs to be a strategic part of the business. However, during the last year and a half, things have moved on and at pace as artificial intelligence seems to be dominating the corporate headlines and not always with positive effects. So today we'll talk about how to make sure that AI can offer benefits to your organization and those within it, rather than just using it to reduce cost. So, Johan, welcome back.

Speaker B: It's good to be back, John.

Speaker A: Yeah, it's good. It's just that 18 months since it disappeared in a blink of an eye, I have to say. Let's start by trying to understand if CRO and AI can work together. But before we do that, I think there's still some confusion in the market about CRO being a complex thing. So do you think that AI will make it even more so, or is the perception a reality in that AI can do everything?

Speaker B: Uh, there's a few questions in there, but let's start with the last one. Can AI do everything? Um, there are two things I know for sure. The one is AI cannot do everything today, but it keeps on getting better, and it can do a lot. And it can do a lot more than we, I think, want to admit to ourselves that it's able to do. That I know for sure. Um, and then I think we can make the safe assumption that this is the worst it'll ever get. But then the second part of that question is, everything. When you say, can I do everything? The goalposts keep on shifting. Um, and so I think that everything will become something that we can't even begin to understand today. We will be able to do a lot more. How do I know that? Because we are able to do a lot more today than we were able to do a year ago, two years ago. And let's not forget, we were able to do a lot more than we were able to do 20 years ago. And that's just, you know, that's. Yeah, that's what this thing allows us to do so it's not as simple as can it do everything? Uh, it will, but it will change.

Speaker A: So interesting you just mentioned there that, um, it's as good as it's going to get. But do you not think the next iteration of say the version 4, as opposed to version 2 of ChatGPT is going to offer even more kind of or less complexity and easier to use and get into the weeds a bit more?

Speaker B: Yeah, thanks for checking because I didn't mean to say that, uh, let's be clear about this. What I mean to say is that today the models that we have is the worst it will ever be. In other words, it will get a lot better. So this is already pretty good. I mean it's phenomenal what is available today. But, um, even if no further development is done, then we still. And that's unlikely, right? It is going to develop into the future, but just based on what we have today, we still figuring out how to get the most out of that. And so I think it's without a doubt it will get a lot better than we have today.

Speaker A: Do you believe that AI and CRO could work in harmony?

Speaker B: Yes, I think AI can work with anything and everything in harmony is the idea that I get. Um, and I think that for me, I mean there's this debate and I'm not sure if this debate is even still. It feels a little bit superficial and like we've moved beyond it. But AI versus human, um, if that ever was the case. I think what I'm increasingly finding the more I use it, is that it's AI plus human, not AI versus human. And so I think that it can live in harmony. And um, if you, it's who uses it, that for me is really the. One of the key realizations about AI is you and I can be doing exactly the same task with exactly the same AI. We'll get different responses, we'll get different outputs. Um, the reason for that being our different backgrounds at different domain knowledge, uh, you know, what we know about the particular area, product, whatever it is. And so I think, um, if you pair it with a specialist, then you'll get very good results. And um, I think that's the way to use it.

Speaker A: So in a recent presentation you did, I can't remember when it was but a few months ago now, um, talk about how AI is a tectonic shift. Can you explain what you mean by that? Because obviously people think it's different and it's uh, it's, it's got Lots of potential in it, but tectonic shift is enormous. It's a big phrase.

Speaker B: Yeah. And I'm not sure if that's big enough. I think you listen to a phrase like that and it's sort of just a, just a phrase. I don't have the language to explain how big I think this is. Um, and what really makes me sit and from the first day made me take notice is that the people on Twitter for example or X who talk about this aren't the, you know, it's not the hype bros, it's serious names. You know, these are people with gravitas, uh, who are making statements like this. But one way I think about it is I would assume that most people listening to this grew up with the Internet. Um, you and I know that things were very different before the Internet. Uh, but while the Internet, you know, in the 90s when the Internet started becoming a thing every day, you could see the advances. You had a sense that this is big. You had a sense that it would change the world. You even saw it on a day to day basis how email came and suddenly you have this magic thing called email on your computer and you could talk to people without having to phone them and pay for the call and um, stuff that we take for granted now and then at some point we look back and you go, hang on, things have radically changed. The world looks very different than it did in 1990. Uh, industries have completely disappeared and others have formed. And so I think that for me is what I'm trying to express is that there's this slow motion shift that's happening underneath us and we, we not maybe we have a sense of it but at some point we wake up and we look back and we go, uh, things are really. The world looks entirely different.

Speaker A: Yeah, I think one of the things is that lots and lots of people are talking about it for various different reasons. It's not just uh, I suppose like the Internet kind of as you say, evolved over time. It seems like in the tech industry we've been talking about machine learning AI for quite some time. But actually now it's hitting the headlines for all probably the wrong reasons. But it seems as be that it's like bang, it's there. And even when I talk to friends around a coffee, it's always about oh, you know, I saw this or I saw that. Oh God, um, I can't believe that they captured somebody's face that quickly. You know, it's in deep fake and that's what it. So it seems to have a, what I would describe as a pretty immediate impact. But one of the other comments you made, I'm not sure if it's in this presentation or another presentation I saw, was that AI can help you move shit around the page, which doesn't help. Most people seem to think that it can. So what's the reality?

Speaker B: I think there's a more fundamental, um, point there, which is that the definition of CRO, um, and what is often paraded as CRO is what I refer to as simply moving shit around the page, you know, so it's this concept where you look at um, a web page or a screen, a mobile screen, as this UI shell, and you move things around, you know, so there's this classic thing of the button color or that's become almost an industry joke. But what I see is, for example, something as superficial as move the button up the fold, you know, it's little superficial rules like that. And so I think it's for me that the point there is about ar, uh, how CR is being defined. And if that's your definition of CRO, then that's how you're going to channel AI. Um, and I think it can absolutely help you do that, that I think you'll be invaded by AI. It will take over. You don't. And AI should be doing that if that is, you know, your definition of zero. And then. But if you think of CRO as a more strategic, and this applies I think to every domain, not just CRO, if you think of it as a more strategic, uh, undertaking. Um, and even, even if you don't see it that way, if you are the doer today, if you are the person moving stuff around the page or whatever the equivalent in your industry is, then we know for a fact that AI will take over that mundane the road tasks. Where does it leave you and what will you be doing in future? Um, and I think that's a really important question to start thinking about if you haven't done so yet.

Speaker A: Yeah, because, because I think that it's um. Some of the quotes that we hear from Sam, say Sam Altman's predictions of uh, CE, who is the CEO of OpenAI, and I quote, it'll mean that 95% of what marketers use, agencies, strategists and creative professionals for today will easily, nearly instantly and uh, at almost no cost be handled by the AI. That's a pretty strong statement to make, isn't it? Do you think?

Speaker B: Let's also remember that Sam Altman also promised us AGI in 20, 25 and that didn't happen. But I think he's right. I think that's um. And again it goes back to the previous answer is I think he's right. Absolutely. I think it's happening already. And whether it's 90% or 95 or 80% that shift is happening. But it doesn't mean that. What it means is that the role of an agency changes and so it moves up that value chain and it becomes more strategic. And if you were previously the doer and if you were previously so in the ad industry for example, which is not my field, but if you're setting up campaigns and uh, that can now be done by the meta AI, um, and so I think even creatives M and copy and the little components. So I think he's right in that sense that those day to day tasks, the execution, the doing of the stuff, the implementing of the strategy, assuming there is one AI will take that over. But I think what the bit that's missing from his statement is that the roles will change, are changing.

Speaker A: Mhm. One of the comments that when I've talked to other people outside of the industry, um, you know, talking about AI and how it's having, how it's being implemented and outside such a dramatic effect is they say that you know, even if AI isn't well say it's only 60% accurate because we all know it's not truly 100%. Won't that be seen by senior management as an excuse to cut back on potential people like CRA specialists?

Speaker B: I think senior management will cut back on uh, CRO specialists and everyone not just CRS specialists. Regardless that is happening and let's deal with that separately. The question about accuracy of AI, um, couple of things to say about that. So that's the so called hallucination, the tendency uh, of AI to make up stuff, um, to manufacture facts and present those facts very confidently when in fact and they don't exist. And there have been some interesting case studies especially with lawyers citing uh, casework, case law where those cases didn't exist and uh, obviously ChatGPT dreams it up. But I think it's become a lot better not I think I know it's so that is less of a problem than it used to be and let's assume that improves as well. Second factor is that see it as a feature, not a bug. That creativity when you write copy, for example, assuming the facts are honored and you're not making up claims that uh, aren't accurate, you've got to check that but that um, tendency of it to hallucinate and to make up stuff, that's actually what you want, you've got to use that in your favor. And the third variable that I think is important to mention is that humans also hallucinate, um, and also, you know, make stuff up when in fact it's um, maybe not based on evidence.

Speaker A: So um, we used to talk, I suppose we still talk about it to a degree about good um, marketers as T shaped marketers. Um, and they have a broad understanding of everything but a specialization of one element. Do you think that's going to be a thing of the past?

Speaker B: That is a very good question. And I, of course, I don't know, um, nobody knows what will happen. And as we saw with the Internet, roles completely disappear and new roles, um, come into its place. And we don't know today what that will be. But the way I think about it at the moment is that the T shaped marketer is still relevant and maybe even becoming more relevant and becoming more dominant. The word that gets used now that I like is full stack marketer. And what that means is that you're able to do, or you should be able to do more of the end to end process than you were able to do before. So in a simple example previously you might be the analyst and you'd hand it off to the designer, the designer would hand it off to the developer and so on. And what AI allows you to do, enables you to do theoretically is to own um, more of that process. And so I think that the T shape model is still a good one to use because what it says is that you've got to have your specialism, you've got to own something that you know really well, your domain, you're something that uh, that's your superpower. And then you've got the T at the top, well the horizontal leg at the, or bar at the top. And I think that's becoming broader. So it's, so there's more in that bar at the top than there used to be before. And also it's becoming fatter. In other words, your knowledge of, of those adjacent areas are perhaps, you know, you require a little bit more um, knowledge about areas that you weren't that familiar with before.

Speaker A: So do you think that means that you've got to have not just a broader aspect but also a deeper aspect as well? But that, that always seems to me that it's not a specialization anymore, it just becomes a jack of all trades. Excuse the pun, but it's not Necessarily, um, you've got, you talk about a specialist because they're defined in their field, they maybe have an educational background in that sphere. And now where we're talking about these people having to have a broader aspect, but a very deep, broad aspect, it could be construed actually that means, you know, is it something that really is getting less and less important and the technology can do more? And uh, just to take that on a little bit further, I understand in smaller organizations that's probably the status quo today because they can't afford vertical specialists who do several things. So there might be one or maybe two people that are doing that. But in larger organizations where you've got people like analysts, content writers, graphic creators, psychologists, surely that must mean that there's going to be redundancies.

Speaker B: Surely again, I think there will be redundancies. It's already happening. There are many reasons for why those redundancies will happen. And um, management and boards will find various excuses associated with AI, but it's just the, you know, that's the. Like it or not, I certainly don't like it. It's not the game I want to play, but it is what's happening already. If you go on Reddit and you um, see um, people in, for example, UX research, so these are people with PhDs, they are people who come from social science backgrounds. They are legit, they are specialists. They not somebody who just from one day to the next called themselves a researcher. They are trained in this. And there's a long thread that gets added a few dozen comments every day of people saying, I've lost my job, I've been, um, made redundant, um, because of um, AI is the reason that's sometimes overtly given to it, but sometimes it's just sort of assumed. But your question about the different roles so that I think will happen, um, the one thing that I've sort of underestimated but has become a big realization for me is that AI in the hands of a specialist, that's really what you're aiming for. So I'm not a developer. I don't come from a coding background. I'm able to do some coding with um, GPT and Claude and all the rest. I'm able to build apps that work that I wouldn't have been able to do two years ago. So that's amazing. But if I show that to an engineer or the kind of problems, for me to get to that point where it's, you know, where it works, it takes hours or an entire weekend, an engineer will look at that and be able to adjust the prompt and direct the AI to get that resolution much faster. We'll be able to solve the problem that I'm unable to solve with, uh, the AI. And I think you can. So I've seen that, um, personally, and what I've also seen is that in something like CRO and I, I've seen this a lot, is that you give the AI and a good brief to a junior CRO strategist and what comes back is underwhelming and below average and not something that I will, uh, uh, want to be associated with. And so what that means is that you give it to somebody who has the domain knowledge, who has the background, who has the experience, who has the expertise, and you get a very different result. And so it's how you use the AI, but how you, uh, the knowledge that you have that you can't even put on a piece of paper. If I say to you, what is that knowledge that is in your brain that you were able to get to this different result which is so superior to the other result? You don't know because it's just, um, tacit knowledge. It's soaked up in your brain cells from having been involved in this over many years and having studied this. And so, John, I think that is the most important realization of all for me so far is that it's um, yes, on the one hand it allows you to step into other areas that you weren't able to do before and you become of that full stack. But the specialist human becomes even more important.

Speaker A: Because if I look at two of the, um, specialist areas, and you could call them left brain and right brain almost, you've got the data analyst role, which has always been very, very analytical, very black and white to a degree. And on the other side of things you've got the psychologist who has got more of the emotion, if you like, the ability to be able to make things sound better and be heard better. And that's a, that might be a funny way of putting it. Do you think that the, uh, balance of power in the new CRO talent stack is probably more towards the psychologist and less towards the data analyst?

Speaker B: No, I don't think so. I think it, um, goes both ways. And again, I think the simple fact here is that the data analyst will be able to do magic in his or her domain with AI and the same for the psychologist. But the psychologist trying to do data analysis with AI will be able to do more than they were previously, um, capable of doing. But it won't be at the same level that. That is certainly where we are today. I think what's also important to say here is, you know, there's. There's this, uh, very, um. I think it's now famous, this, um, cognitive bias, the Dunning Kruger effect. And what the Dunninger Kruger effect says is that we tend to overestimate our competence in a field that we not very familiar with. So something that's not really our domain, as we start learning about it, we very quickly get to a point where we think we got this. But actually that's far from the truth is when you compare your knowledge to a real specialist, there's a massive gap. And I think that is the danger that is happening with AI right now is that if I'm a psychologist and I'm doing some data analysis, uh, to use your example, that the AI gives me a confident response when in fact there's maybe a better response. But I don't have the domain knowledge to know that there is a better response, to know that I asked the wrong question. I don't perhaps even have the vocabulary to ask it for the right output. And then I think that's the real danger is that we, when you put the Dunning Kruger effect and AI together, it's a very unfortunate combo.

Speaker A: Interestingly, I had, um, some friends around yesterday and one of them was a psychologist and I kind of posed the question that we were going to be talking about this today to a degree. And I said, do you think that psychologists are concerned about AI in any way? Because lots and lots of the other industries are very concerned about AI. And she said, I think it will have an effect, but she's not quite sure how it's going to have an effect. And she's basically sitting watch this space. But she feels that the emotions and the ability to be able to listen for that, her particular role, which obviously is face to face or at least face to screen these days, um, won't be affected for a while. But she doesn't. She's not naive enough to think it's never going to affect her because obviously it'll affect lots. One of the things that we did talk about actually yesterday was do you think that AI should be rewriting the job specifications for people within the organizations? Because obviously there's got to be some level of trimming. And I think that's an interesting, um, kind of topic we ought to cover. What would we get out if we threw some questions into the mix? Because I know it's going to be different for Every business. But it just seems like that AI, I won't say has all the answers, but certainly by many will be seen as the place for the answers. Would you agree with that?

Speaker B: Yeah, I would agree with that. I think it's hard for me to think of any task or subtask in my role where AI does not have a role to play, where it isn't relevant and writing job descriptions is just one of them. Um, I cannot think how I would go back to a time where I would be doing something like writing a job description or whatever else is the equivalent without the help of AI. Um, but of course, again, it's how you use AI. And so what I will never outsource to AI is my thinking and my direction. It is the way I use AI in that context. And in fact I think most contexts is, um, clarifying my thoughts, helping me to express something better, challenging me. So, you know, is there something that I'm missing here? And, and then also that the more your domain knowledge you give it, the more context you give it, um, the better it's able to advise you. As long as you don't ask AI to write the entire thing for you. Um, I think that's it is the way to use it.

Speaker A: Yeah. Just taking a step forward. I know you've got, um, children like I have. Fortunately, my children are already through the educational process and at the other end and have all got jobs. But what would your recommendations be for an individual at university, perhaps with an eye on becoming a marketer? Do you think they need to be specialists? And if so, where do you think is that it's the best place to put their money?

Speaker B: Yeah, you spot on. I mean we having those conversations right now because my kids are about to, um, you know, exit school and start thinking about university and what they do next. And I don't know, um, is the answer. I do think that, um, there's a, to your point about humans being social animals and wanting to, you know, we need to interact with people. Um, so, so how do we lean into that, um, and, and try to understand what tasks will be, are likely to be completely invaded, taken over by AI. But also remember the calculator story, you know, in the 70s when the calculator became a thing, they were worried. Well, I wasn't around to say the headlines, let's be clear about that. But I heard stories on the Internet, um, on my mobile phone that um, there were worries about what would happen to accountants. And of course we know, uh, it just gave accountants a different role and so then Excel came, and now we're able to do calculations. In my game, for example, we do calculations that we take for granted that 20 years ago we didn't even have the computers, we didn't have the chips to be able to do those calculations. It's done within seconds now. Um, so I think that the way to think about this, and it's really. I don't know the answer, but it's to think of a role and a task as two separate things, and a role is made up of tasks and subtasks, and which of those tasks and subtasks are likely to be taken over by AI. So that's the one thing. And stay away from them. Um, and then, of course, the humanness, being human, how do we lean more into that and how do we make more of that? Because I think that's the one thing that hasn't changed for thousands of years and won't change, because that is how we are wired.

Speaker A: Yeah, I think the calculator analogy is a very good one, and it did touch a lot of people's lives, having a calculator to be able to do basic arithmetic much, much more quickly. I think the fear of AI is that people don't know what it can't do, what the possibilities can't be. And so being able to try and come down a solution and say, actually, uh, for my career, I want to be this. It's very difficult when you're sitting there doing your A levels or just got your A level results in the last few weeks to be able to say, actually, yeah, I think what I was thinking about doing may be something I ought to be thinking differently. And one of the things I propose, and I'm not sure whether they're there yet, is there, um, a course to be a specialist AI marketer? And if so, you know, how does that, um, seem? Because obviously we're changing very quickly into this. There's some people that have been talking about AI for quite some time, but I think academia seems to be, uh, at the pace of commercialism, and academia seem to be a bit slower than if you were in pure academia. Because obviously, I know AI is doing tremendous things in university research, but what kind of knowledge are our youngsters having to be able to gain, to be able to try and make those decisions, to put themselves in a role for the future? I just don't know.

Speaker B: See, I've got a different view on that. I. And again, the only thing I know for sure is that I'll be wrong. Um, we don't know how this is going to play out. But the, the way I think about it today based on everything I've seen so far and experienced is that the uh, it won't be an AI marketer, it'll be a marketer with AI. It won't be an AI academic, it'll be an academic using AI. And so I think that's the, my thesis is that it's the specialist married coupled with, paired with the AI. Um, and maybe that doesn't apply universally. You know, maybe there are roles that, where that simply doesn't hold. But I think at least for the immediate future that's the way I see it playing out.

Speaker A: So if you're already in marketing in a role somewhere and you've been a content writer, uh, or a graphic designer, where do you think or what do you think they should be doing now to be able to broaden that skills?

Speaker B: They should be using AI. Hopefully they are. Hopefully by the time you listen to this, you that that's not necessary to say anymore. Embrace it and figure out how it is able to empower you. You know, how it lets you do things that you weren't able to do before, how it lets you do things at a better quality, how it lets you do things that um, you didn't even think were possible before. Because that's, that, that's the real what is your value and how you know if it's not writing words, if it's directing the thinking that becomes the words. So in my world that's very real. You know, copywriting CRO is a very important part because the way you say things and the actual words that you use matter and makes a difference to conversion rate more than many other um, variables. And so that starts with the way it plays out in my world for example is it starts with doing consumer research, looking at customer reviews, talking to customers, understanding what happens in the mind of the consumer, how they think, what are their pain points, how do they think about this, what are the mental models, what are the actual words that they use? How do they describe this problem? What are the pre purchase anxieties? What is that moment where they decide yes, I need to buy this thing and then taking all of that and transforming that into a sales conversation. Now if you asked AI uh to write a headline for a web page of an E Commerce web page, it'll do it. But if you gave it all this information and you applied your mind and your models and what I've just said in a very uh, small summary. But there's a lot of detail behind that. You get a very different outcome. And so I can't remember what your question was, but again it's that how we use it um, is often forgotten in the way we superficially talk about

Speaker A: AI Ah, well, I think in the short space that we've been talking about A.I. uh, not just in this conversation but over the last few years, it's as we said, it's much more of a revolution than an evolution. I think. It's not evolved, it's kind of appeared almost suddenly I would say.

Speaker B: I think John, what I'm able to do today wasn't possible a month ago.

Speaker A: Mhm.

Speaker B: And I was able to say that every month for the last six months things. And that's coming back to the earlier question about tectonic shift. Things happen in slow mo. But if you look back you will see just how fast things are moving and how far we've gone.

Speaker A: Well, I can't wait for the next six months. It's obviously going to be far more exciting or perhaps it's just the realization of what we're already thinking anyway. But for now, Johan, it's been a great conversation as it was the last time and I think we've got plenty to talk about in six months or another 18 months time. With regard to what happened. How's the market changed? You know, has E Commerce totally moved on in a different way than it has been today? Today? What was the relationship between E Commerce and uh, on site stores? Is it going to go back to on site? Who knows? We'll have to have that conversation at some extra time. But for today, Johan, thank you very much for your time. It's been very, very interesting.

Speaker B: John, very good to talk to you again. Thank you. Mhm.