Series 6 Episode 7 - Can experimentation help your organisation achieve sustainable business growth?

Speaker A: You're listening to the Big Lift, the podcast of WebTrends Optimize, the CRO solution that enables marketers and developers to maximize the ROI on their digital properties. WebTrends Optimize is a powerful feature rich and easy to use solution, all delivered within a fixed price contract with no additional cost for increased functionality ever. During these podcasts, we meet some of the key influencers within the marketing and conversion world to understand their roles and examine their challenges.

Speaker B: Today I'm welcoming back Johnny Longton, who first appeared on the Biglift podcast nearly four years ago, back in September 2021, where we discussed, uh, how profitability in research should drive CRO. As you'd expect, things continue to move at pace in experimentation. So today Jonny returns to talk about building a strategic framework for achieving sustainable business growth. So, Jonny, firstly, many thanks for being a guest this morning and welcome back to the Big Lift podcast. I can't believe it's been four years since we last chatted.

Speaker C: It's fast. It doesn't seem like that long.

Speaker B: It doesn't, though. Anyway, um, about this discussion. The term experimentation led growth, or elg, is quickly entering into the vocabulary of successful businesses. But can you explain what ELG really means?

Speaker C: Yeah, um, so I think the background is that, um, experimentation has really kind of come to mean a B testing. And a lot of businesses, whether or not they consciously think like that, they have a setup where they think, we've got an A B testing tool, we've got somebody who can run it, therefore we do experimentation. And that's sort of a legacy thing from CRO. It's just how things have been in the past. So a lot of senior people that hold budgets in businesses wouldn't really question that. Uh, but as soon as you pick it apart, it very quickly doesn't make any sense.

Speaker B: Yeah.

Speaker C: And the analogy that I use, I'm doing quite a bit of public speaking around this at the moment. The analogy that I use is the Apollo space mission. So if you think about that, you know, it's arguably the greatest achievement of humankind.

Speaker B: I'm not sure everybody wrote. I'm not sure everybody will remember it though, because it's many, many years ago.

Speaker C: Yeah. From a technological perspective. But how did that succeed and what was it? Well, first of all, it was a bold and audacious goal and very clear goal to get a man to the moon and return him within a decade, which was set by jfk. And then the entire endeavor was a very highly, uh, coordinated program of work involving around 400,000 people. That succeeded because of the program management. So you know, the leaders of that program would have taken that goal and figured out how do we coordinate all these people to test all of the different things and to learn and to figure out how we do this in a coordinated fashion that brings all of that learning together into um, a series of iterative test flights until the goal is achieved. So in that sense the whole endeavor was experimental, um, it was run experimentally. And that is incredibly different from saying that it included an experiment or some experiments on say rocket propulsion. And if you go back to modern businesses, even some of the most advanced businesses in terms of experimentation are ultimately simply giving um, access to a lot of people within the business to experimentation tools and to the ability to experiment. But if you imagine that in that space program, imagine they'd taken that goal from jfk, got a load of people together and said go off and experiment on, you know, anything that you can think of wouldn't have got very far. It was the coordination and the management, the systems management that brought all of that together. That is what allowed it to succeed. And so the term experimentation is located in the wrong place. In businesses today it is thought of as a tool, a uh, technique to, to do something and not as the way to fundamentally strategically innovate the business. So that is what experimentation led growth means. It is trying to use the broader concepts of experimentation to innovate and grow the business whilst coordinating these activities across different functions. So there are some fundamental things. Number one, it is the belief that experimentation does not need to mean a B testing. You know, there may not have been very much controlled experimentation going on in that space program, but it doesn't make it any less of an experiment. So uh, it doesn't need to mean AB testing, um, it's about gathering evidence, uh, and being as confident as you possibly can. Um, so that's one thing. The second thing is that it's about coordination and about um, learning um, in ways that the whole business can benefit from. So there are some interesting developments um, in the last 10 years where particularly SaaS companies coming out of um, Silicon Valley are organizing themselves differently and, and they have uh, growth models where there is a VP of growth and marketing products and customer operations line under growth. And that gives the ability for that centralized growth function and that senior growth function to be coordinating these experiments that might span different functions. So ultimately there, the principle is that experimentation does not need to mean front end website content which is, is where it's being pigeonholed into. So yeah, that's that's a kind of a high level summary of it. But you know, at the end of the day it's trying to locate the concept of experimentation in a very different place in a very different um, mind space to how it's been um, thought of in the past, which is in, in tooling and techniques and things like that.

Speaker B: So I can see it with startups and west coast startups are very, very agile but for an established business that's really I would think quite challenging to be able to set a singular um, goal if you like, or even multiple goals which they can then garner the whole team, the whole company together to make that achieve. Because there's so much kind of historical stuff that's been going on. Do you think it works as well in existing businesses?

Speaker C: It is challenging, uh, but it's not impossible. However, um, the fundamental thing is that it really relies on more senior people being involved in the transformation. So that's the nub. Um, you know a lot of um, a lot of big companies have experimentation teams. They have product teams that are running experiments and the people running these programs inherently understand the value of what I'm talking about. There is the, there is an enormous frustration experimentation functions around the world that they are not being, not delivering the value that they could deliver. And that is because most of these people understand that what I'm talking about here is ultimately how it should work. That experimentation should be treat as a strategic function, not as a tactical, um, you know, optimization function. But unfortunately the transformation required to do that is going to require C suite people. So I personally think that we as a industry of consultants and anybody else who is charged with helping progress this sort of world need to really kind of focus on more senior individuals within the business. Uh, because you're absolutely right, it is challenging. But it comes down to um, number one, the structure of the organization and the way the organization works together and collaborates and coordinates. But number two, how objectives and how vision and mission is communicated into those functions. So you know, uh, you rightly point out that ah, it's difficult to come up with a single vision. Well, um, you know, it doesn't have to be a, it doesn't have to be as clear as let's put somebody on the moon. But any business should have a strategic direction. They should have a strategic vision that can be translated into smaller projects in exactly the same way. You know, I've developed some quite interesting ways of doing that and we at SPIRO help our clients do that a lot in terms of how we uh, ladder of, you know, big objectives and big strategic initiatives down into manageable, chunkable things that can be tested and uh, hypotheses that can be learned from. So again, it's absolutely not impossible, but it does require more senior involvement. If you have somebody, you know, if you have somebody who's historically been running a B testing in an optimization function, they're going to struggle to suddenly start to um, transcend that world into like a very, very strategic business function.

Speaker B: And that's, that's one of the challenges, one of the biggest challenges I think, because experimentation as we know it five years ago is very much within the marketing, the data and analytics and things like that. And this strategic initiative, as you say, is sitting at the board level. And to be Honest, in the UK now we don't have many directors or VPs of growth. Maybe in tech companies we do, but outside of that we've got a cmo, you know, basically who runs the kind demand generation for the business. And we might have a sales VP if it's a, if it's something like a B2B organization. But that's quite a different perspective because you've got to bring so many different teams along with a Director of growth, I suppose you've got to have the marketing, you've got to have technical, you've got to have data, you've even got to have sales, you've got to have uh, I suppose that strategic presentation of this is where we want to go and guys, you're all coming with us. So in the absence of having a Director of growth, who would you see would be the best person to lead this from a, a new opportunity inside a business which currently doesn't use um, ELG at all.

Speaker C: Yeah, it's going to vary from business to business, but I think, um, probably, you know, a Chief Commercial Officer would um, be the most sort of traditional role that would be able to spearhead and lead it because ultimately it is about revenue growth and about um, the commercial side of the business. So a Chief Commercial officer is in a lot of businesses, probably the spearhead. You do have revenue officers, so a lot of businesses have a chief Revenue officer, which is more or less the same as a chief growth officer. Other than that, you know, an operations chief operations officer, um, is potentially looking at that because, because ultimately it's about how you coordinate the business together. So you know, a lot of businesses are incredibly siloed with um, you know, with marketing and product and things like that. And they uh, you know, big businesses, they can have very Little interaction with each other at all. But it's the chief Operations officer that, you know, really ought to be thinking about how to, you know, uh, break down those silos and to ensure that people are operating and working in the right way. So yeah, in different businesses it will be different. These roles aren't the exact same things in the different businesses, but it's going to be somewhere around there.

Speaker B: Would you kind of start this um, ELG kind of initiative in one department or is it a C suite conversation that you have to have?

Speaker C: First of all, I think that it's not going to get very far without a C suite conversation. Um, because the problem that we come up against again and again is a lack of understanding, uh, from the senior layers of the business. The unfortunate thing is that experimentation has a long and very established history and people, without really properly thinking about it, have been exposed to the idea of experimentation over many years. So anybody coming up through a marketing function or having, you know, worked in any kind of reasonably sized business of any kind, will have become aware of the concept of a B testing or CRO or things like that and they will have the very wrong perception of what it is. So, you know, most people believe that CRO is um, like PPC or SEO or something like that. It's a channel, it's something that you put money into and get money out. It exists in a P and L line item, um, and it's simply a kind of, ah, a cost and return kind of thing where you know, you're, you're just generating incremental revenue. It's generating incremental revenue in and of itself, um, and can tick along by itself whilst you get on with other more important strategic things. So whether whether or not people think that consciously or not, that is, what is, what is in the back of their head about what experimentation is. Um, there's a completely different view of experimentation which comes from the startup world where, you know, there's a really, really generic idea of experimentation that simply means trying things. Yeah, so, you know, people who have built startups will have done that through a process of trial and error of just trying different stuff, um, but without any kind of system or rigor to it. So, you know, it's just a really general notion of the fact that you're kind of figuring stuff out as you go along. So somewhere in amongst both of those things, people have got really incredibly

Speaker B: set,

Speaker C: um, idea of what experimentation is. And so if you, anyone trying to kind of make these philosophies, uh, kind of known in the business is going to come up against that and really you need to kind of reset what people understand. I would say the single kind of most important thing that anyone can try and get across is that experimentation is an operating system. That's what most of the C suite will not really understand, um, is that they either think of it as optimization or they'll think of it as just generally trying stuff. But the central notion of ELG is that it is an operating system, it is a way of working, is a, uh, framework, it's organizational structure. You know, it's all of this kind of stuff that has to be brought together and put together for it to work in exactly the same way as the Apollo space program. You know, they would have had to think really hard about how experimentation in different areas is feeding and learning into the overall program. There's a, there's an interesting story actually, about how, um, the testing with rocket propulsion, um, creates vibration in the rocket. So, um, you know, and anyone trying to develop any kind of telemetry machinery or anything else to do with the rocket would have to be completely in tune with the frequency of vibration that the rocket propulsion was creating. Because if you're creating instrumentation in the cockpit, the frequency of vibration is important to how you create it in such a way that it won't fall apart. So all of that learning has to be fed across different areas of the business. And everybody has to be in tune with what everybody else is doing. And so that requires a system of management and government and uh, learning and collaboration. And that is what experimentation is. That's the thing that's missing. That's the thing that these businesses are not getting. They might say, oh, we run some experiments, but all it means is let's try doing this thing and see what happens. It's not coordinated, it's not organized, uh, that sort of stuff. So experimentation is an operating system is the message, and that is what we have to try and help people build. Um, and then of course, experiments will happen within that.

Speaker B: Uh, yeah. Do you think that just like the experimentation world as we know, a B testing, CRO, the naming conventions of those three things all meaning similar things with regard to testing, Your website has, I think, caused a lot of confusion in people to think, well, is this AB testing? Is that the same as CRO? Is CRO the same as experimentation? Do you think perhaps ELG is poorly named because it's jumping on the bandwagon? There's something that's already there. Whereas if you, um, business process re engineering, which was around in the 90s and 2000s was very much of a similar ilk. Although not using experimentation as its lead. It was being able to understand about how the business works and then designing structures inside of that. Would you say that ELG is similar to that, or is it just, um, something that's new that's going to, um, be adopted? Because people are always looking for something that's going to generate more revenue, especially these days.

Speaker C: Yeah, I mean, uh, I mean, from the point of view of our work at Spiro, we don't tend to use the term ELG, um, for the reason that there are lots of LGs, you know, there's product led growth and there's all sorts of things like that. And it's a little bit, you know, I think probably a lot of people would just generally roll their eyes at like another lead growth sort of thing. Um, we're using the term growth experimentation, uh, which is, I think, clearer. I think it's, you know, it says. I mean, it's very similar, obviously, but it is clearer. It's, you know, it's. It's the idea of growth, which is a. Is a popular term at the moment, and the idea that you would run growth through experimentation. But the semantics of it are incredibly challenging. Um, I don't have like a perfect answer. It's. It, uh, is an unfortunate thing that the history of CRO and a B testing has gone the way it has because, um, you know, it is. It is a really fundamental part of why it's misunderstood. Conversion rate optimization is the most misleading term that there is in business. Um, you know what, what you actually do in that world doesn't have to be about conversion rate, doesn't have to be about optimization, and it might not optimize your conversion rate at all. Conversion rate is a very complex, very ambiguous metric that, uh, is influenced by literally everything about the entire business market, competitor, everything. It's just a pure piece of maths. Um, and what you do when you run experiments might have no impact on that whatsoever because, um, you know, there are so many of the forces impacting it. So conversion. The word conversion rate optimization makes no sense whatsoever. Um, but unfortunately it's kind of stuck. And that's a big part of the reason why a lot of people think of it as, oh, a B testing, a B testing software. We're improving the conversion rate. It's a tactical thing. And it never ceases to amaze me how much m. How little people understand what conversion rate is and, uh, how to understand fluctuations in it, you know, I've been in roles in my career where a big part of my job has just been continually trying to explain that to people and trying to, and trying, going on wild goose chases, trying to explain small fluctuations in it and things like that. Um, so, yeah, that, that has caused a lot of problem. But I don't know, the, the, the growth area is a, uh, real step forward in that because it's such a clear expression of what you're trying to do. You know, you are simply trying to grow. Like you can't think of that without thinking about innovation and pivoting business models and things like that. So growth is a real interesting step forward. If experimentation can be aligned to that, it makes it much clearer for anybody instantly thinking about it as, you know, what are we doing? We're experimenting in order to grow. We're experimenting in order to figure out how we, uh, go on a journey with the business and to change.

Speaker B: So you're trying to pitch your way as a spiro into an organization. What is it that your key message to somebody, you've got to have some reason to be able to pick the phone up, as they used to say, or at least connect with somebody. Now I know you do a lot of presentations around the place. There's lots of noise around this, but how do you get that, I don't know, CMO or growth officer to be able to answer, um, your call or answer your email. It's a challenging perception to get across the line. And I think from a sales perspective, if you're in a B2B, that must be really difficult to be able to succinctly get that message across to the person who's going to help implement this. So, uh, how do you get managed to make that work?

Speaker C: Yeah, well, it's, um, There is no one simple thing that works for absolutely everybody. Um, you know, different businesses are different. The person you're talking to is different. The challenges they have are different. Ultimately though, like, you know, in a hypothetical situation where I happen to be in an elevator, I've got, you know, 30 seconds with the chief growth officer of a SaaS business. You know, what I would be saying is experimentation, um, is an operating system. That's what I was saying. That's the fundamental thing that we're trying to just spark somebody to think about, um, is, you know, yes, you might be doing experimentation, yes, you might be, you might be trying things, but have you got a rigorous system about it? Have you got a, uh, coordinated, organized way of running experiments and managing how they're run almost definitely not. And that experimentation, operating system will help you to grow, it will help you to systematically grow. So the way you say that, the way you get that message across obviously is, is different in different cases, but that's ultimately what we're trying to get people to understand that, you know, it isn't just optimizing, tweaking things. Um, so, but you're right, it is challenging. Um, there's two sort of approaches is one to try and speak to those people, ah, you know, directly. The other is a more kind of account based marketing approach to try and um, bubble the conversation up through people that are already in the business. Um, which is a lot of, you know, thought leadership kind of content and just trying to, you know, provide support in thought leadership and things like that.

Speaker B: So you get through to this guy and he says, yeah, that's, that sounds really good. What's the first steps that you have to take as a business to be able to start yourself or your company on this road to experimental growth?

Speaker C: Yeah, so I would say number one is kind of auditing. So, you know, exactly like I said, every business is different. Uh, what is the nature of the business at the moment? How, how are strategic decisions made? Uh, how are different functions organized within the business? Are they very siloed? Are they working together? Is there a growth function? So just simply auditing and understanding the business as it pertains to how innovation works and how decisions are made and how growth happens. And then number two is to, is to just simply get stuck in with an example. So, um, any business will have big things that they are thinking about, like, you know, should we expand into new geographies, should we um, create this new product, should we start charging for returns, you know, whatever. So, uh, any business will have these big fundamental things that are being discussed and pondered in the boardrooms and you can just take one of those and start running it through this process. So, you know, that's kind of what we help people do. I've got a good example, like I was working with a really big, um, department store retailer years ago and they had this really fundamental question as to how they appeal to a younger audience. Um, and uh, you know, that's quite a challenging thing to sort of try and approach. But it becomes quite easy when you break it down. Like if you're going to appeal to a younger audience, that is going to happen through one of three things. One, you're going to fundamentally change the product that you're selling or at least change how you're merchandising the product and how you're expressing who you are through product. Two, you're going to start marketing and targeting people in addressable media, um, more aggressively so that you can target the right people. And three would be about the nature of the brand, messaging, tone of voice and all that kind of stuff, the way you speak to people. So each of those three, that very big sort of ambiguous idea of how do we speak to younger people? Can be broken down into three, three more manageable things. And those things you can already see are easier to test. Like, um, should we change the product? You can change the products that you're merchandising on, landing pages, emails and marketing and advertising and measure whether or not that's pulling in a younger audience, you know, so on and so forth. So each of those things can be broken down into little manageable tests. And again, it's the idea of how you get people to go off and learn those things, bring that learning back so that you can start to see. Well, actually it seems like we're getting traction with this idea of merchandising, visual merchandising and where we place our products. Let's burrow into that more and figure out exactly how we do that. So, you know, you start to evolve a strategic theoretical idea of how to solve this problem through, um, experimentation. So that's what we'd always encourage companies to do, is to just pick one thing and to just start doing it and just start figuring out how to work. And through doing that as well, you would then expose silos and ways that things don't work together and also be able to develop theories about how perhaps the business needs to evolve and change.

Speaker B: Does it need strategic goals to be able to make that happen or is that you can do it from the ground up?

Speaker C: Yeah, I mean, you can do it either way. Yeah, like, um, you know, um, there can be small ideas that would blossom into bigger things. Um, I was working years ago with a bed company and you know, we found they sold different types of beds in terms of where they came from. So they had their own, um, warehoused products that had very fast delivery. And they also drop shipped other manufacturers beds that they had to manufacture and send, which had a lot of long delivery time. And we found just through basic kind of customer insight that people wanted fast delivery. And so that became a bigger strategic project to try and understand what would happen if we didn't sell these drop ship beds at all. It would fundamentally change the nature of the business. Like, you know, would For a start, be able to kind of almost rebrand the businesses like Express Breads and things like that. And it would change SEO, uh, it would change everything. So that small bit of insight that we found became a big strategic project. So you know, any, it can come from anywhere like that. But um, you know, they have to be what we call big bets. You know, you want to, you want to be um, you know, looking at how you're like really sort of, you know, pivoting the business model in some way, the brand in some way, because that's what's going to drive breakout growth. You know, you're not going to get breakout growth from just tweaking UX on the front end of a website. You know, you want to be whatever it is that you're trying. It needs to be something that is, you know, really kind of changing the shape of the business or how it operates with customers or things like that.

Speaker B: One of the things of doing initiatives like this is that there tends to be a lot of talk around it. So more meetings, more um, collaboration between parts of the organization to make these things happen. Do you think there's capacity in businesses today that because so many of them are fighting for profitability?

Speaker C: Capacity to what, sorry?

Speaker B: They've got the capacity to be able to have more meetings and more discussions or make this kind of shift change to doing something different. Because at the moment, if you talk to many retailers especially, they're all about fighting profitability. How can they generate not just revenue but actually bottom line profit?

Speaker C: Yeah, that's where the idea of an operating system comes in. Um, you know, an operating system, what we mean by that is A1 level, some kind of project management system, like literally the software that allows you to break uh, objectives down into different things and to track those and to run them and to prioritize them. Much like a, you know, like a prioritization system in a, in a more kind of um, siloed CRO function but on a broader level. So that should be the starting point for the organization. Understanding where the priorities are and built within that should be okrs or you know, whatever it is that you use to be able to track objectives and to convert strategy and vision into manageable things should sit within that. Ah, so that should in theory take away a lot of admin because you can quite clearly kind of farm um, tasks out to individual people that, that are contextually coordinated within bigger projects and things like that. But you know, to your exact point, profitability could be the starting point of that. So um, you Know, profitability could be the most important strategic driver. And then the question becomes, how do we drive increased profitability? Um, and again you're looking for like the big bets. You're looking for like, what if we um, start charging a lot for uh, returns? Now that is a big thing. That's a big change to the way you operate with customers. You see recently what ASOs have done, it gets a lot of public, um, interest in that sort of stuff. So it's a big fundamental change to the nature of the business model and the interaction with customers. So that is a big bet. And you know, hopefully ASOS kind of tested that. But um, you know, those sorts of things are um, um, are those big bets that you can kind of drive strategically. So as long as the operating system exists and that, you know, there is coordination around how these kind of things are, there doesn't need to just be endless meetings of people sort of separately trying to figure out this problem. Because that's the whole point is you're coordinating people to participate in these bigger ideas that have been sort of mandated, uh, as priorities.

Speaker B: And what tool do you use to manage those multiple projects along the way? So you've got one objective and inside that objective you've got 10 or 15 other things that have to be in place to make that objective a reality. What kind of is it an Excel spreadsheet or is it something more sophisticated or are there tools out there?

Speaker C: I think that's a really unique challenge for each individual business that there isn't like a one size fits all. I mean, um, things like Airtable are really well placed to uh, create these systems because it's going to be bespoke. Each business is different and the way it runs is going to have to be different. So you kind of have to create bespoke system. The challenge with Airtable is that uh, doesn't necessarily integrate particularly well with execution systems like Jira and other marketing systems and things like that. If you want that better integration, there are sort of product road mapping tools and other similar tools like that that have better integrations. But um, there are dedicated strategy management software tools and things like that. So uh, it's. There is no one answer that like all, you know, anybody should use. It's a kind of a very specific question for each business to think about how their systems work and how their data works and all that kind of stuff. Because you know, what you really don't want is to be duplicating information and having people copying and pasting and things from one System another and stuff like that. So um, yeah, there'd be some, there are some principles that you can create around how, you know, there needs to be a centralized management of initiatives that can trickle into other systems that um, you know, the way it works and the way that sort of filters down. But the system itself, um, you know, is a question for each business on its own.

Speaker B: One of the things that I've kind of experienced in businesses that I worked for, there's a very structured business which gives you the kind of, the confines of which you need to do your job, make things happen and those people that, for want of a better word, allow people's individual thought processes to work through. How do you try to create a balance so that you don't have the kind of the culture that develops into um, ask forgiveness later than ask for permission because that might be good or it might be bad. But in having something that isn't kind of structured down to a level. So if I use your analogy with regard to the Apollo mission, um, when it comes to being able to say, well actually we need to understand what surface the moon is to understand what kind of pads we have under the landing module, they're not really going to know that success until actually they land on the moon, basically. But somebody's got to be able to find some way of being able to I suppose gauge what that success would look like. And I think people could go off and do lots and lots of what I would describe is potentially useful but also potentially time wasting effort. And they think that they're doing this for the good of the project but without kind of structure into it. And I'm thinking that this uh, experimentation led growth is actually a very good thing because I think there are lots of things we learn from experimentation in our daily lives. You try and if it doesn't quite work, we tweak it. We do something else. If you've got kids, you're doing experimentation every day of the week. You're experimenting what works and what doesn't work. So I think the culture is very good, but it's. I find that trying to get that down to the troops inside the business to be able to refine what they're doing into a place that is heading towards that goal might be a little challenging, do you think?

Speaker C: Yeah, I mean staying on the uh, Apollo mission analogy, it's a good one by the way. Yeah. So staying on that analogy, like you know, the wood be there would be a team, let's say, developing um, this space suit technology and they would Have a very clear goal that came from the overall program with wider contextual information about this is the expectation of how long we believe the people will be on the moon. Like, this is the expectation of how, of like the heat that they will go through when they go through the, you know, all that kind of stuff. So we'll have that contextual information to be able to then, um, you know, design a space suit that um, works and that, and that fulfills those objectives. But at the same time, um, you know, they should be able to uh, pass learning back so they might find out something completely random. Some completely, you know, um, whilst investigating this, we've discovered something about. I'm completely making this up, but like, you know, we've discovered something about the way, um, the heat will be likely to change or something like that. And we need to feed that back into the overall program. So there's a sort of a bottom down, bottom up and top down combination of things that needs to happen. And I think that translating that into business from a cultural point of view, it's a important that both of those things work. That a team has a clear objective of something that they're working on that's part of a coordinated approach of other things. And yet they are also free to explore, um, you know, things and to find out random things and to pass that into either the same program or into new ideas and stuff like that.

Speaker B: So just one final question as we're getting close to the end of it. Generally speaking, in experimentation, CRO, whatever we call it, there's always, uh, an anticipation that success can be measured. And in what this thing, this broader aspect of experimentation, I think it can be difficult to be able to have a finite measure of the success. Because even if you do what you expect to do, you've grown your market share in, um, younger audience by 20%. You say, well, why couldn't we do it by 25%? Or uh, what were the learnings that we got from that? So you'd go back and keep doing experimentation upon experimentation like we do in CRO. So it's an evolution rather than evolution, would you say?

Speaker C: Yeah, so I mean, this is a really important point and I think one of the most severe limitations, a B testing is that it relies on a metric for its success outcome and metrics are incredibly myopic. Um, and the business world is kind of obsessed with metrics. But metrics are very limiting and they only tell you one thing. So, uh, you know, best example I can give of that is that, uh, in the retail world, for example, you can very easily increase your conversion rate, revenue, and probably profitability temporarily by reducing your prices or applying heavy discounts. Um, and it will work like you would positively impact all of those metrics, which are fundamental business metrics. However, you would be potentially destroying your business in the long term through creating a poor perception of the product, through, you know, creating a perception of the price as cheap, um, uh, locking people into only wanting to pay you very heavy discounts, all that sort of stuff. And all of those things cannot be measured in metrics. Uh, similarly, you know, you might enter into a kind of a price war with a competitor that can't be made into a metric. But those things are really important and they are having a genuine impact on your business. But the obsession with metrics completely ignores all of those things. And so you've got businesses really like, on kind of downward trajectories without realizing it, because all they can see is metrics. So, and, you know, there's no way that the space program was run on a series of metrics. Um, impossible. So the answer, without going into it, is systems theory. So, you know, really we need to kind of embrace systems theory that understands these broader effects that changes have that can't be made into metrics and can't be made into numbers. But nevertheless, you can still understand them intellectually and track them and understand what, what things are going to do. So I think that's really a fundamental part of this idea of growth, experimentation, and the idea that, you know, you're not necessarily running a B tests that you have to have. You have to find ways of being able to test things and trial things and understand what is this going to do to brand perception, to customer loyalty, to customer advocacy, to all that kind of stuff, none of which can be made into a nice simple metric and yet is very real.

Speaker B: So, Jonny, I think this is, as we know, it's an evolution, as I said, rather than a revolution. And that evolution takes time, takes understanding, and takes a broader knowledge than what's currently out there. And it's obviously from your perspective, Sparrow is doing a great deal of education along the way to be able to make these things more people more aware of these type of things. I think the next two or three years, depending upon what happens in the States and the rest of the world, that's going to give us the opportunity to look back on this. Hopefully not in four years after the last one we last spoke just to see how well all this is starting to progress and started to make traction. But for Now, Johnny, thank you very much for a very interesting conversation. And I. I'm going to be watching this space.

Speaker C: Great. Thank you very much.

Speaker B: All right. Thanks a lot, Sam.